Lutfur Rahman was banned from standing for office for five years in 2015 after being found guilty of electoral fraud.
It is understood Mr Rahman is starting the party with his former deputy Ohid Ahmad who is bidding to be elected at the 2018 mayoral election.
Mr Ahmad said "everyone has the right to start a new political party or to support and existing one".
Campaign literature featuring both Mr Rahman and Mr Ahmad appears under the banner of the "Tower Hamlets Together" party.
The same name and logo also appear on an application sent to the Electoral Commissioner to set up the new political party.
The government has sent a letter to the commission asking it to "forensically review" any application for a new party involving Mr Rahman.
In the note, Minister for the Constitution Chris Skidmore MP said he "would be grateful if you could indicate" if a disqualified individual "should be involved with the registration of a political party".
In a statement, Mr Ahmad said he was standing in the 2018 mayoral election but had "yet to finalise whether I will stand as an independent or under a party banner".
A group of voters took legal action against Mr Rahman in 2015 over a series of allegations of election fraud including ballot paper tampering.
Mr Rahman was removed from office and the 2014 mayoral election was declared void after he was found guilty of corrupt and illegal practices.
He maintained there was "little, if any" evidence of wrongdoing against him and later unsuccessfully challenged the ruling of the Election Commissioner Richard Mawrey.
Current Mayor of Tower Hamlets, John Biggs, who won a new election in June 2015 for Labour says Mr Rhaman "clearly hasn't learned".
"I'm hopeful the people of our borough will recognise that he left us in a dead end and that we need to move forwards" he added.
The disgraced former mayor of Tower Hamlets is trying to form a new political party, the BBC understands.