
how can >this possibly be fair to the infants?

What do you mean fair?  God is just, giving to everyone what they
deserve. As all infants are in sin from the time of conception (cf
Romans 5.12, Psalm 1.7), they cannot possibly merit heaven, and as
purgatory is for the purging of temporal punishment and venial sins, it
is impossible that origianl sin can be forgiven.  Hence, the unbaptized
infants are cut off from the God against whom they, with the whole of
the human race except Mary, have sinned.  Which is why Jesus said,
"Truly, truly I say to you, no one can enter the Kingdom of God unless
he is born of water and Spirit" which is the true meaning of born again
(John 3.5).  Thus, as infants are in sin, it is very fair for them to be
cut off from God and exlcuded from heaven.

As St. Augustine said, "I did not invent original sin, which the
Catholic faith holds from ancient time; but you, who deny it, without a
doubt are a follower of a new heresy."  (De nuptiis, lib. 11.c.12)