Khan was born in Rajasthan into a Muslim family of Pathans with Rajasthani ancestry. Khan's mother, Saeeda Begum Khan was from Tonk, and his father, Yaseen Ali Khan, was from the Khajuriya village in Rajasthan's Tonk district, and they ran a tyre business. He spent his childhood in Tonk, and then Jaipur. Irrfan was good at cricket and was selected to play in the CK Nayudu Trophy for emerging players in the under-23 category, a tournament seen as a stepping stone to first-class cricket in India. However, he did not attend as he could not afford travel expenses.