His term as Speaker was not without controversy, and when he adjourned Parliament to delay debate on a Bill that his party opposed, a vote of no-confidence was held on him in May 1973, passing by 21 votes to 15. However, with the constitution requiring a two-thirds majority for a Speaker to be removed from office, he remained in the chair. He later resigned as Speaker in 1976, only to become embroiled in the emerging conflict within the NFP. Differences within NFP were patched up for the elections held in March 1977, but R. D. Patel stood against Sidiq Koya as an independent candidate. Although he lost the election, he won a significant number of votes and "softened" Sidiq Koya for the first defeat of his political career six months later in the September elections.