The output of the amplifier will not drift from zero when there is no input. The capacitance (C) of the capacitor and the input and output resistances of the stages form an RC circuit. This acts as a crude high-pass filter.
The output of the amplifier remains constant when there is no signal, which is because of the capacitor's role in creating a high-pass filter with the input and output resistance.