Stiller was the oldest of four children, Stiller was born in Brooklyn, New York City, the son of Jewish parents, Bella (nee Citrin) and William Stiller, a bus driver. His paternal grandparents immigrated from Galicia, and his mother was born in Poland.