Serge Poltoratzky was the only son of Dimitry Poltoratzky and Anna Khlebnikova, who also had five daughters. Serge was primarily educated by tutors at the family home, Avchurino, on the east bank of the Oka River in Kaluga Province ("Kaluzhskaya Oblast"), but he also spent a year at the Richelieu Lyceum in Odessa. His parents were deeply interested in improving the material and social conditions of Russian serfs and peasants, and Serge inherited their commitment. In 1812 the family hid in Avchurino's attics as Napoleon’s army looted the estate during their retreat from Moscow.