At a BBC debate, Russ Timpson called for a conversation about how many people could live "on this island".
The Tory, UKIP and Labour candidates on the panel said they understood him to mean limits on having children, respectively describing his comments as chilling, astonishing and "barmy".
The Green Party said austerity had placed the country under pressure.
All five candidates had been asked if Britain was full, by an audience member at the Radio Kent debate in Margate.
Mr Timpson, a Liberal Democrat candidate in South Thanet, said there were too many cars and too many people, leading to pressure on schools and A&E, and he described a manifesto produced by Population Matters - which promotes "a sustainable family size and mindful consumption" - as "getting somewhere".
But when quoted by other candidates as favouring limiting the number of children people had, he denied it and said he had called for a population growth debate.
UKIP candidate for Rochester and Strood Mark Reckless said England was now the most densely-populated country in Europe. He called for controls on the pace of people coming in so young people could have opportunities and there could be a "more comfortable" pace of change.
Dover Conservative candidate Charlie Elphicke also said migration had led to too much social change in too short a time and proper border controls were important - but he accused UKIP of talking about "bongo bongo land and ting tongs" and said there had to be a "serious debate".
Labour candidate in South Thanet, Will Scobie, said the answer to the "strain on resources" was to tackle tax avoidance, reintroduce fairness into the immigration system, and grow the economy in an inclusive way, and not to adopt policies "out of a science fiction film".
South Thanet Green Party candidate Ian Driver said pressure was not from immigration but rather down to austerity and cuts by both Labour and Conservative governments. He said the country faced a housing crisis going back 15 to 20 years.
Dover candidates
Rochester and Strood candidates
South Thanet candidates
Calls for a debate on population growth have come from a Kent Liberal Democrat election candidate.