Personnaly, I would use Theano flags :) I don't know if it should be added directly in Theano. Do tensorflow support this? If it is added in keras, I would do:  ``` mode = None if _DEBUG_MODE == 'detect_nan':     mode = 'NanGuardMode' ```  Can you open an issue on Theano, Using the Theano flag should not enable the GPU. I don't have this behavior on the lstm example. 