title: How to set a constant as a weight in a keras model?
body: <p>I build my model using <code>tf.keras.layers.Dense</code>. In the first layer of my model I want some weights to be constant Zero. As in the gradient calculation these weights should be get a gradient = zero (as the last term in the chain rule corresponds to the weight, which is 0 for a constant).
  This is my approach so far:</p>
  
  <pre><code>import tensorflow as tf
  import tensorflow.contrib.eager as tfe    
  import numpy as np
  
  tf.enable_eager_execution()
  
  model = tf.keras.Sequential([
    tf.keras.layers.Dense(2, activation=tf.sigmoid, input_shape=(2,)),
    tf.keras.layers.Dense(2, activation=tf.sigmoid)
  ])
  
  weights=[np.array([[tf.constant(0), 0.25],[0.2,0.3]]),np.array([0.35,0.35]),np.array([[0.4,0.5],[0.45, 0.55]]),np.array([0.6,0.6])]
  model.set_weights(weights)
  def loss(model, x, y):
    y_ = model(x)
    return tf.losses.mean_squared_error(labels=y, predictions=y_)
  
      def grad(model, inputs, targets):
    with tf.GradientTape() as tape:
      loss_value = loss(model, inputs, targets)
    return loss_value, tape.gradient(loss_value, model.trainable_variables) 
  </code></pre>
  
  <p>But in the gradient calculation the weight tf.constant(0) has a gradient not equal zero.
  Do I have an understanding problem?</p>
  
  <p>How can I set <strong>a</strong> weight(or <strong>some</strong> weights) in a layer(<strong>not all</strong> weights in one layer) to a constant value (which should not change during training)?</p>