The watchdog said just 54% of primary age children in Coventry attended a school rated good or better, compared with a national average of 74%.
Eleven primaries are due to be inspected this week, alongside phone surveys at other schools.
Coventry City Council said improvements had been made over the last year.
It said, since November the proportion of children attending a good or outstanding school had risen from 42% to 58%, slightly higher than figures contained in Ofsted's annual report, because of more recent inspections.
The local authority manages 89 primary schools in the city.
Councillor David Kershaw, responsible for education, said he was confident local primary schools would continue to improve.
"There is still more work to do and we have identified a clear strategy with head teachers and governors to achieve this," he said.
Ofsted said the inspections were being carried out to find out why there were differences in standards between local authorities with similar demographics, such as population and levels of deprivation.
The watchdog said it hoped the week-long inspections would give "a powerful snapshot" of how primary schools are performing since its annual report was published.
It warned, depending on what inspectors found, the local authority could be investigated over its statutory duty to provide fair access to good quality education, under new rules due to come into force in May.
Mr Kershaw said the council welcomed the latest inspections and that it was on "a secure journey of improvement to ensure that every school in Coventry is good or outstanding".
Coventry City Council said while there were still improvements to be made at its primary schools, some 85% of secondary school pupils received a good or outstanding education, well above the national average of 71%.
Ofsted inspectors are targeting Coventry's primary schools in a bid to find out why they are 'underperforming'.