<p>In this paper : <a href="https://arxiv.org/pdf/1311.2901.pdf" rel="nofollow noreferrer">“Visualizing and Understanding Convolutional Neural Networks”</a> , Zeiler and Fergus discussed the idea that this renewed interest in CNNs is due to the accessibility of large training sets and increased computational power with the usage of GPUs. They found and also talked about the interesting ways of visualizing feature maps and weights. It is one of the best and prevalent papers in the field of Deep Learning. Writers also emphasized about the limited knowledge that researchers had on inner mechanisms of these models.</p>

<blockquote>
  <p>Saying that without this insight, the <strong><em>“development of better models
  is reduced to trial and error”</em></strong>.</p>
</blockquote>

<p>They also use AlexNet like model in their paper. This model which was named as ZFNET was the winner of  ILSVRC 2013 .I am sure by reading this paper you will have better understanding about overall DL concept and possible solutions to your question.</p>
how to visualize weights between layer convolution?
<p>I have a small deep learning problem. Here I built my network (CNN) with the bookstore Keras. I am interested in visualizing the weights of my CNN. My architecture is AlexNet type and my color images (RGB) are divided into 72 classes. For the first convolution which has 96 filters whose filter kernel is 11 by 11 I recover a 4 dimensional tensor at the output [11] [11] [3] [96]. So each filter has 3 matrix 11 by 11 which we will call kernel.</p>

<p>At this level for the visualization of my weights I took an image I split it in 3 channels. For a given filter each channel was convoluted with a kernel. Each result of these convolutions operations has been gathered to give a resulting image.</p>

<p>Now the second convolution that takes input the output of the first is set with 383 filters whose filter kernel is 5 * 5. The output of this second convolution gives me a tensor 4d of size [5] [5] [96] [383]. This means that for a given filter it has 96 filters (at least that's what I understand). So there for a given filter I'm still with my famous image splitted on these 3 channels facing 96 filters.</p>

<p>I do not know if it is a problem of understanding but I block total because on the output of the second convolution I do not know interpreted the 96 kernels for each filter.</p>

<p>I would like to from my weights reconstitute the filters a convolution layer.</p>

<p>I am really novice in deep learning it is an interesting science but full of mystery for me. If anyone had the kindness to enlighten me I would thank him.</p>