As a non-graphitized carbon material, possessing exceptional hardness and chemical inertness, glassy carbon (GC) is often synthesized through the pyrolysis method, which includes a compression procedure of powdered precursor materials, thus increasing the costs for production of glassy carbon at an industrial scale. Direct preparation of GC via pyrolysis of bulk precursors is a low-cost approach but encounters challenges arising from an insufficient knowledge of carbon structure formation. In order to solve this problem, a new analysis of the temperature-dependent variation in Young’s modulus of GC obtained by the pyrolysis of phenolic resin at 1000 °C, utilizing the impulse excitation technique (IET), was performed. Our findings demonstrate that there is a critical temperature range of 500–600 °C where pyrolysis leads to the most significant density change and GC is formed as a result. When GC samples are heated again, a significant structural reformation occurs in the same temperature range. It causes a decrease in stiffness, especially at heating rates >3 °C/min, and an interesting restorative effect–increase in stiffness when a GC sample is annealed at temperatures of 500–550 °C. These results bring important implications for the direct formation of large amounts of glassy carbon using bulk precursors.