Q:

Filtering a signal to have a constant exponential moving average

Problem
I wish to filter a discrete-time input signal, $x[n]$, so as the output signal's, $y[n]$, exponential moving average, $a[n+1] = a[n] + \alpha(y[n] - a[n])$, tracks around a constant reference $\mu_x$ such that $\mathbb{E}\{y[n]\} \approx \mu_x$, and also an approximately constant variance, $\text{var}\{y[n]\} \approx \sigma_x^2$, while maintaining as much information about the incoming signal as possible.
Naive Motivating Solution
1) Maintain an average of the input, $b[n+1] = b[n] + \alpha(x[n] - b[n])$ and an approximation of the signal variance, $c[n]$.
2) Define the output to be $y[n] = \frac{x[n] - b[n]}{\sqrt{c[n]}}$ which tracks a reference of 0 with unity variance.
Question
There are a number of problems with this approach which make it not very numerically robust, such as when low values of variance occur.
Is there a more intelligent way of doing this? Perhaps using a dynamic linear filter? Or high pass filter?
Other Points
The inputs can be considered independent, and are drawn from one of several dynamic distributions.
The application, if you are interested, is comparing the amplitudes of recent inputs with each other by assigning them a "reward" output. This is high reward for relatively higher input values and a low for lower lower values. However, I desire the mean reward to be an approximately constant output mean. Furthermore, there may be some medium-time dynamics in the inputs, such as decaying toward zero over time.

A:

I'll draw on my (limited) experience in audio mixing and mastering. Speech recorded through a microphone requires the same statistical properties (constant moving average and constant variance), and two straightforward processing stages ensure this.
An "exponential moving average" is another word for a 1-pole low-pass filter. This moving average can be brought close to zero by passing the signal through any high-pass filter, such as a "DC killer" or "remove subsonic rumble" filter that an audio processing package such as Audacity or Audition may provide. The stronger the high-pass characteristic, the closer the moving average will stick to the center line.
Second, you want to make the variance constant. That's a job for a level compressor. Find the RMS (square root of variance) of the signal over the next few milliseconds, and adjust the gain (amplification) of the signal over that period to bring the variance up or down to the variance you want. Audio processing packages provide this as well.
You mention numerical stability. The standard DSP literature describes ways to implement a robust high-pass filter. For instance, FIR filters tend to be very stable.

