Buying a house has a huge opportunity cost. Many who bought homes missed other investment opportunities (some of which performed even better than real estate... such as buying Facebook stock after their IPO. Google, Amazon, many any of the big tech stocks outperformed real estate as an investment).

But even worst, many home buyers missed the opportunity to live life, because they were house rich, cash poor. I personally know home buyers who forewent family vacations, or even delayed having children. That's a huge non-monetary cost in order to participate in the housing market.