It’s interesting to speculate on this stuff as lay people, for sure.  My kids have always been interested in space, and often flummox me with questions like “If the universe is infinite, how can it be expanding?”

The best I can offer is that while black holes consist of unbelievably dense matter, that matter still has to occupy space…..and if more matter is sucked into them, they would get bigger.  

Quite honestly, these kinds of questions start to hurt my head sometimes, and I have been known to surmise that theoretical physicists operate on the same plane as Zen Buddhists – contemplate unanswerable questions like what is the sound of one hand clapping in order to clear the mind………