<blockquote>
  <p>how to interpret the weight histograms and distributions in Tensorflow?</p>
</blockquote>

<p>Well, you probably didn't realize it, but you have just asked the 1 million dollar question in ML &amp; AI...</p>

<p><em>Model interpretability</em> is a hyper-active and hyper-hot area of current research (think of holy grail, or something), which has been brought forward lately not least due to the (often tremendous) success of deep learning models in  various tasks; these models are currently only black boxes, and we naturally feel uncomfortable about it...</p>

<blockquote>
  <p>Any good resource for it?</p>
</blockquote>

<p>Probably not exactly the kind of resources you were thinking of, and we are well off a SO-appropriate topic here, but since you asked...:</p>

<ul>
<li><p>A recent (July 2017) article in Science provides a nice overview of the current status &amp; research: <a href="http://www.sciencemag.org/news/2017/07/how-ai-detectives-are-cracking-open-black-box-deep-learning" rel="nofollow noreferrer">How AI detectives are cracking open the black box of deep learning</a> (no in-text links, but googling names &amp; terms will pay off)</p></li>
<li><p>DARPA itself is currently running a program on <a href="https://www.darpa.mil/program/explainable-artificial-intelligence" rel="nofollow noreferrer">Explainable Artificial Intelligence (XAI)</a></p></li>
<li><p>There was a workshop in NIPS 2016 on <a href="http://nuit-blanche.blogspot.gr/2016/12/nips2016-interpretable-machine-learning.html" rel="nofollow noreferrer">Interpretable Machine Learning for Complex Systems</a> </p></li>
</ul>

<p>On a more practical level:</p>

<ul>
<li><p>The Layer-wise Relevance Propagation (LRP) toolbox for neural networks (<a href="http://www.jmlr.org/papers/v17/15-618.html" rel="nofollow noreferrer">paper</a>, <a href="http://www.explain-ai.org/" rel="nofollow noreferrer">project page</a>, <a href="https://github.com/sebastian-lapuschkin/lrp_toolbox" rel="nofollow noreferrer">code</a>, <a href="https://github.com/VigneshSrinivasan10/interprettensor" rel="nofollow noreferrer">TF Slim wrapper</a>)</p></li>
<li><p>FairML: Auditing Black-Box Predictive Models, by Fast Forward Labs (<a href="http://blog.fastforwardlabs.com/2017/03/09/fairml-auditing-black-box-predictive-models.html" rel="nofollow noreferrer">blog post</a>, <a href="https://arxiv.org/abs/1611.04967" rel="nofollow noreferrer">paper</a>, <a href="https://github.com/adebayoj/fairml" rel="nofollow noreferrer">code</a>)</p></li>
<li><p>A very recent (November 2017) paper by Geoff Hinton, <a href="https://arxiv.org/abs/1711.09784" rel="nofollow noreferrer">Distilling a Neural Network Into a Soft Decision Tree</a>, with an independent <a href="https://github.com/kimhc6028/soft-decision-tree" rel="nofollow noreferrer">PyTorch implementation</a></p></li>
<li><p>SHAP: A Unified Approach to Interpreting Model Predictions (<a href="https://arxiv.org/abs/1705.07874" rel="nofollow noreferrer">paper</a>, authors' <a href="https://github.com/slundberg/shap" rel="nofollow noreferrer">code</a>)</p></li>
</ul>

<p>These should be enough for starters, and to give you a general idea of the subject about which you asked...</p>

<p><strong>UPDATE</strong> (Oct 2018): I have put up a much more detailed list of practical resources in my answer to the question <a href="https://stackoverflow.com/questions/52391871/predictive-analytics-why-factor/52392344#52392344">Predictive Analytics - “Why” factor?</a></p>
How to interpret weight distributions of neural net layers
<p>I have designed a 3 layer neural network whose inputs are the concatenated features from a CNN and RNN. The weights learned by network take very small values. What is the reasonable explanation for this? and how to interpret the weight histograms and distributions in Tensorflow? Any good resource for it?</p>

<p>This is the weight distribution of the first hidden layer of a 3 layer neural network visualized using tensorboard. How to interpret this? all the weights are taking up zero value?</p>

<p><img src="https://i.stack.imgur.com/5Xaqa.png" alt=""></p>

<p>This is the weight distribution of the second hidden layer of a 3 layer neural:</p>

<p><img src="https://i.stack.imgur.com/Ux9Tg.png" alt=""></p>