It comes after a prisoner was stabbed to death and two others were injured in London's Pentonville prison on Tuesday.
The PGA called Tuesday's incident a tragedy, but said it was "no massive surprise" given the cuts to staff and funding,.
The government said new plans for prison safety and reform would be set out in the coming weeks.
Addressing the Lords at question time Justice spokesman Lord Keen of Elie dismissed the call for a public inquiry.
"It is not thought that a public inquiry would be the way forward when we are about to publish a white paper on prison safety and reform, in which we will address these issues," he said.
The family of Jamal Mahmoud, a 21-year-old of Somali descent who was killed on Tuesday, hit out at the prison for "neglecting him".
There have been several warnings about prison safety in the last year.
Pentonville was last year singled out by former Justice Secretary Michael Gove as "the most dramatic example of failure".
People living near Pentonville prison have called on the police to take action following what they say is a rise in criminal gangs smuggling drugs into the jail.
Former chief inspector of prisons and independent crossbench peer Lord Ramsbotham said the stabbings at the prison drew more attention "to the fact that our prisons are currently in crisis".
He said the call for an inquiry from the "very reputable" PGA amounted to a "vote of no confidence in the years of purely in-house tinkering with the system by successive ministers and officials".
Lord Keen said the government was determined to modernise the prison estate and tackle drugs and violence.
The government had already announced an extra Â£10m for prison safety and would deploy 400 extra staff by March next year, he said.
The Prison Governors Association (PGA) has demanded an inquiry the state of jails in England and Wales.