Even Jesus recognised there were sheep in the fold who should not be there:

"But because you are not My sheep, you refuse to believe. My sheep listen to My voice; I know them, and they follow Me. I give them eternal life, and they will never perish. No one can snatch them out of My hand."

Those predestined to salvation (yes, the Catholic Church teaches this) will never be lost to Christ.