 <p>One solution is to create a version of your network that is truncated at the LSTM layer of which you want to monitor the gate values, and then replace the original layer with a custom layer in which the stepfunction is modified to return not only the hidden layer values, but also the gate values.</p> <p>For instance, say you want to access the access the gate values of a GRU. Create a custom layer GRU2 that inherits everything from the GRU class, but adapt the step function such that it returns a concatenation of the states you want to monitor, and then takes only the part containing the previous hidden layer activations when computing the next activations. I.e:</p> <pre><code>def step(self, x, states): # get prev hidden layer from input that is concatenation of # prev hidden layer + reset gate + update gate x = x[:self.output_dim, :] ############################################### # This is the original code from the GRU layer # h_tm1 = states[0] # previous memory B_U = states[1] # dropout matrices for recurrent units B_W = states[2] if self.consume_less == 'gpu': matrix_x = K.dot(x * B_W[0], self.W) + self.b matrix_inner = K.dot(h_tm1 * B_U[0], self.U[:, :2 * self.output_dim]) x_z = matrix_x[:, :self.output_dim] x_r = matrix_x[:, self.output_dim: 2 * self.output_dim] inner_z = matrix_inner[:, :self.output_dim] inner_r = matrix_inner[:, self.output_dim: 2 * self.output_dim] z = self.inner_activation(x_z + inner_z) r = self.inner_activation(x_r + inner_r) x_h = matrix_x[:, 2 * self.output_dim:] inner_h = K.dot(r * h_tm1 * B_U[0], self.U[:, 2 * self.output_dim:]) hh = self.activation(x_h + inner_h) else: if self.consume_less == 'cpu': x_z = x[:, :self.output_dim] x_r = x[:, self.output_dim: 2 * self.output_dim] x_h = x[:, 2 * self.output_dim:] elif self.consume_less == 'mem': x_z = K.dot(x * B_W[0], self.W_z) + self.b_z x_r = K.dot(x * B_W[1], self.W_r) + self.b_r x_h = K.dot(x * B_W[2], self.W_h) + self.b_h else: raise Exception('Unknown `consume_less` mode.') z = self.inner_activation(x_z + K.dot(h_tm1 * B_U[0], self.U_z)) r = self.inner_activation(x_r + K.dot(h_tm1 * B_U[1], self.U_r)) hh = self.activation(x_h + K.dot(r * h_tm1 * B_U[2], self.U_h)) h = z * h_tm1 + (1 - z) * hh # # End of original code ########################################################### # concatenate states you want to monitor, in this case the # hidden layer activations and gates z and r all = K.concatenate([h, z, r]) # return everything return all, [h] </code></pre> <p>(Note that the only lines I added are at the beginning and end of the function).</p> <p>If you then run your network with GRU2 as last layer instead of GRU (with return_sequences = True for the GRU2 layer), you can just call predict on your network, this will give you all hidden layer and gate values.</p> <p>The same thing should work for LSTM, although you might have to puzzle a bit to figure out how to store all the outputs you want in one vector and retrieve them again afterwards.</p> <p>Hope that helps!</p>