You are correct as far as you go.  You neglect the part where He loved the entire world so he is drawing the entire world to him.    Again you are denying the aspects of the Bible where people make choices about following him.  When the Israelites left  Egypt, where they not the "Chosen" people, the Elect, as you would have it?   What happened when Moses went up to receive the Ten Commandments?  Did they not CHOOSE to make a false god?  So again choice is involved.
2 Peter 1:10 Therefore, brethren, be even more diligent to make your call and election sure, for if you do these things you will never stumble; 11 for so an entrance will be supplied to you abundantly into the everlasting kingdom of our Lord and Savior Jesus Christ.
Does this verse sound like there is no choice involved?  People were not created to be robots.   Reread the Bible  again, this time notice the times where choice is involved.     Also note that the elect that you go on about are those that choose to obey.