His dismissal came after blasts in Tianjin killed at least 135 people.
The authorities began investigating Mr Yang last week because he allegedly allowed companies to operate without a licence for dangerous chemicals.
But there has been no word to suggest his sacking is linked to the disaster.
The industrial accident earlier this month was one of the worst in China's history.
China's top prosecutor is reported to be investigating all involved officials for dereliction of duty and other crimes.
Thousands of people in the city saw their homes either destroyed or made too unsafe to live in.
The blasts took place at a warehouse storing hazardous chemicals in Tianjin's port, and required a massive rescue operation and clean-up involving thousands of police and soldiers.
It is still unclear what caused the explosions.
The two owners of the plant are alleged to have used their connections to obtain safety licences to store hazardous materials, even though the site was too close to residential areas.
Correspondents say that the breaches of law and discipline that Mr Yang is accused of are generally used by the governing party as a euphemism for corruption.
At least 34 people are still missing while more than 500 are being treated in hospital, Tianjin officials say.
Mr Yang, 61, worked in the city for 18 years and became one of its vice mayors before beginning employment at the work safety agency in 2012.
Industrial accidents occur frequently in China, correspondents say, with corruption considered to be one of the main reasons for inadequate safety enforcement.
The blasts provoked uproar among local residents, with many alleging that government collusion with the firm led to pollutants contaminating Tianjin's air and water. The city is home to about 15 million people.
Mr Yang's son, Yang Hui, is also being investigated by the authorities, the South China Morning Post reported. It said that he was well connected in Tianjin because of his father.
China's work safety regulator Yang Dongliang has been sacked by the Communist Party for suspected "serious breaches of discipline and the law", state news agency Xinhua has reported.