<p>It is not that simple. For example, in later stages the variable could be reduced to 0.</p>

<p>I'd have a look at <a href="https://arxiv.org/abs/1606.05386" rel="noreferrer">LIME</a> (Local Interpretable Model-Agnostic Explanations). The basic idea is to set some inputs to zero, pass it through the model and see if the result is similar. If yes, then that variable might not be that important. But there is more about it and if you want to know it, then you should read the paper.</p>

<p>See <a href="https://github.com/marcotcr/lime" rel="noreferrer">marcotcr/lime</a> on GitHub.</p>
Is there any way to get variable importance with Keras?
<p>I am looking for a proper or best way to get variable importance in a Neural Network created with Keras. The way I currently do it is I just take the weights (not the biases) of the variables in the first layer with the assumption that more important variables will have higher weights in the first layer. Is there another/better way of doing it?</p>