Ullman was born in Eilat, Israel, to American Jewish parents Laura (née Ehrenkranz), a teacher, and Brian Ullman, a printer. He has a younger sister and younger brother named Tali and Nadav, respectively. After his first birthday, his family left their kibbutz in Israel for Norwalk, Connecticut, United States, and then to Fairfield, Connecticut the following year. When Ullman was younger, his father worked as a clown. While attending summer camp, he played Peter Pan, which helped him discover his talents in acting, singing, dancing, and reading drama novels.