Govorukhin was born in Berezniki, Sverdlovsk Oblast (now Perm Krai). His parents divorced before he was born. His father Sergei Georgievich Govorukhin came from Russian Don Cossacks and was arrested as part of the decossackization genocide campaign started by Yakov Sverdlov. He had been exiled to Siberia where he died around 1938 at the age of 30. His mother Praskovya Afanasievna Glazkova was a tailor. She came from the Volga region, from a simple Russian family of a village school teacher. She raised Sergei and his sister Inessa by herself and died at the age of 53.