<p>For precisely two reasons:</p>

<ul>
<li>There is no one sequence of compiler optimizations that can simultaneously optimize all possible program characteristics of interest, such as execution time, compilation time, code size, energy consumption, binary-portability, etc. In compiler optimizations research, this is known as the phase ordering problem.</li>
<li>Most developers do not want to bother with figuring out which compiler optimizations to use and in what order; they just want to use whatever is generally recommended in a small number of common scenarios.</li>
</ul>

<p>That's why compiler developers have decided to offer a small collection of optimization levels from which developers can easily choose in general, yet <a href="https://hal.inria.fr/inria-00451106/" rel="nofollow noreferrer">offering hundreds of fine-grained optimization options</a> for advanced scenarios.</p>

<p>The term "optimization levels" is really a misnomer, since they are not exactly "levels" with respect to each other. A better term would be something like "optimization groups".</p>

<p><a href="http://users.elis.ugent.be/~leeckhou/papers/cgo08.pdf" rel="nofollow noreferrer">Designing optimization levels</a> is a complicated matter for compilers that target a broad range of programs and architectures, such as GCC, Clang, icc, and VC++. Many research papers have been published in the past decade that show that the optimization levels offered by compilers are far from being the best for a particular program, target architecture, or specific collection thereof. This motivated a line of research on <a href="https://arxiv.org/abs/1801.04405" rel="nofollow noreferrer">compiler auto-tuning</a>, which can be considered as an approach that falls somewhere in between offering few optimization levels and offering fine-grained control over compiler optimizations.</p>

<p>In summary, optimization levels provide an important convenience for developers, which will be required for many decades to come.</p>
Why C++ compilers have many optimization levels
<p>I was just thinking why C++ compilers have many optimization levels like O1, O2 etc. Why can everything be part of just one optimization level O. </p>

<p>I tried search online a lot but didn't got a convincing answer for this.</p>