In her latest report, Noelle Buick said that those from socially disadvantaged backgrounds were particularly affected.
She also expressed concern over how children are taught, pointing out that about one fifth of lessons were "less than good".
However, she acknowledged that a number of areas of education had improved since her last report, in 2014.
The Education and Training Inspectorate (ETI) observed more than 11,000 lessons and training sessions in Northern Ireland over the past two years.
While the majority of early years, primary and post-primary schools were doing well, a significant number of children were "accessing education that is not good enough", she said.
She said that about eight out of 10 primaries and seven out of 10 post-primaries were providing "high quality education".
However, that meant that "just over 78,000 children may not be getting a good enough education".
It cited reasons, including "a variation in standards across individual subjects, low teacher expectations and uninspiring teaching".
While she said most schools had good management, she was critical of the leadership in some grammar schools, saying it was "less than good" in a quarter of them.
That compared to poor leadership in only one in 10 non-grammar schools.
Her report also said it was "unacceptable that boys continue to under-perform" in exams, compared to girls.
In 2014-15, four out of 10 boys in non-grammars got at least five GCSEs at grades A*-C, including Maths and English.
That compared to just over half of girls in non-grammars.
Ms Buick blames the above trend partly on schools having fears "over the position of the school in league tables published by the media [rather than] the best interests of the pupils".
She concluded that, while progress has been made in many areas, improvement will only come with more joint working between government departments and between schools and parents.
Northern Ireland's chief inspector of schools has said too many pupils are not getting a good enough education.