Khan joined the Indian National Congress party and was elected to the Lok Sabha in 1980 from Kanpur and 1984 from Bahraich. In 1986, he quit the Indian National Congress due to differences over the passage of Muslim Personal Law Bill which was piloted by Rajiv Gandhi in the Lok Sabha. He was against the triple talaq legislation and resigned because of differences with Rajiv Gandhi on this issue. Khan joined the Janata Dal and was re-elected to the Lok Sabha in 1989. During the Janata Dal rule Khan served union Minister of Civil aviation and Energy. He left the Janata Dal to join the Bahujan Samaj Party and again entered the Lok Sabha in 1998 from Bahraich. Khan held ministerial responsibilities from 1984 to 1990. In 2004, he joined the Bharatiya Janata Party (BJP) and unsuccessfully contested the Lok Sabha election as a BJP candidate that year from Kaiserganj constituency. Khan reactives in BJP from 2015 after Narendra Modi became the prime minister of India.