<p>Fundamentally, you are changing the number of parameters of your model. As you go from</p>

<pre><code>conv1 = tf.get_variable('conv1_1', shape=(11, 11, 3, 64), initializer=tf.contrib.layers.xavier_initializer())
</code></pre>

<p>to</p>

<pre><code>conv2 = tf.get_variable('conv2_l1', shape=(11, 11, 3, 20), initializer=tf.contrib.layers.xavier_initializer())
</code></pre>

<p>Your learnable parameters goes from {Kernel x 64} to {Kernel x 20}
This will require you to re-train the network and learn its new weights. </p>

<p>However, this is a common problem that has evolved into a research area. Many methods have been proposed for this such as low-rank approximation of weights (Denton et al., 2014; Lebedev et al., 2014), weight quantization(Courbariaux et al., 2016; Rastegari et al., 2016), knowledge distillation (Hinton et al., 2014; Romeroet al., 2015) and network pruning (Han et al., 2015; Li et al., 2017), among which network pruning has gained notable attention due to their competitive performance and compatibility. </p>

<p>References to explore:</p>

<ol>
<li>Emily L Denton, Wojciech Zaremba, Joan Bruna, Yann LeCun, and Rob Fergus.  Exploiting linearstructure within convolutional networks for efficient evaluation. InNIPS, 2014.</li>
<li>Vadim  Lebedev,   Yaroslav  Ganin,   Maksim  Rakhuba,   Ivan  Oseledets,   and  Victor  Lempitsky.Speeding-up convolutional neural networks using fine-tuned cp-decomposition.ICLR, 2014.</li>
<li>Matthieu Courbariaux, Itay Hubara, Daniel Soudry, Ran El-Yaniv, and Yoshua Bengio.  Binarizedneural networks:  Training deep neural networks with weights and activations constrained to+ 1or-1.arXiv preprint arXiv:1602.02830, 2016</li>
<li>Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network.NIPSWorkshop, 2014.</li>
</ol>
How to reduce the number of kernels/filters in a trained model in Tensorflow?
<p>If I have a trained model, where I want to retrain the same model, with few filters/kernel removed from the existing model. e.g.  </p>

<pre><code>conv1 = tf.get_variable('conv1_1', shape=(11, 11, 3, 64), initializer=tf.contrib.layers.xavier_initializer()),
</code></pre>

<p>and I want to resize this tensor such that it has the shape of (11, 11, 3, 20) but the same name and position, mean exactly the same variable. Advance thanks for the help.</p>

<p>I have tried <code>tf.reshape</code> but it gives me error of not matching the number of elements in a and b 
I have also tried <code>tf.assign(a,b, validate_shape=false)</code></p>

<pre><code>self.weights = {
    'conv1_': tf.get_variable('conv1_l1', shape=(11, 11, 3, 64), initializer=tf.contrib.layers.xavier_initializer()),
    'conv2_': tf.get_variable('conv2_l1', shape=(7, 7, 64, 128), initializer=tf.contrib.layers.xavier_initializer())
}
</code></pre>