In modern enterprises with a separation of powers, the ultimate controller can effectively influence the implementation of corporate strategy and operational management efficiency, as well as improve corporate governance by monitoring and limiting the management entrenchment effect within enterprises. Based on the information pertaining to ultimate controllers disclosed by enterprises in their annual reports, this study empirically tested whether the absence of the ultimate controller impacts investment efficiency using the data of Chinese A-share listed companies from 2007 to 2020. It was found that the investment efficiency of enterprises without ultimate controllers is relatively lower than those with ultimate controllers. This is reflected in the insufficient investment of enterprises without an ultimate controller. Moreover, the effect is more significant when the financial environment, internal governance environment, and external governance environment of firms are worse. The mechanism analysis demonstrated that the absence of an ultimate controller causes a more severe insider agency problem and a significantly higher degree of financing constraints, which leads to underinvestment and reduces investment efficiency of firms. The economic consequence test also found that the inefficient investment caused by the absence of ultimate controllers would damage the future value of enterprises, but would increase managers’ compensation. Overall, this study suggests that ultimate controllers are an important part of a firm’s internal governance, especially for monitoring management behavior and resolving agency conflicts.