<p>Though, the concepts may be abstract, they find good use in recent times in Machine learning/Artificial intelligence.</p>
<p>This might serve as a good motivation on practical need for these theoretic concepts. In summary, you want to estimate how well your model  (LSTM, CNN for example) does  in approximating the target output ( using for example cross entropy  or  Kullback-Leibler Divergence from information theory). (check  <a href="https://openreview.net/pdf?id=ry_WPG-A-" rel="nofollow noreferrer">on information bottleneck</a> and <a href="https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=7133169&amp;casa_token=Cuofu0kgJg0AAAAA:JeCJJd34wAMlvYCReIDrJc4RjNokpWs6mnjf1Toz1ffrujgcvf0X7EBMcNO4fuQFEYwsvYBDuiPV&amp;tag=1" rel="nofollow noreferrer">deep learning and information Bottleneck principle</a> for perspectives on explaining deep learning through information theory)</p>
<p>In addition, you won't build a useful <a href="https://rads.stackoverflow.com/amzn/click/com/B00FB33A7Q" rel="nofollow noreferrer" rel="nofollow noreferrer">communication</a> or <a href="https://rads.stackoverflow.com/amzn/click/com/B00EANELDA" rel="nofollow noreferrer" rel="nofollow noreferrer">networked system</a> without some  analysis of the channel capacity and properties.</p>
<p>In essence, it might look theoretic but it is at the heart of the present communication age.</p>
<p>To get a more elaborate view on what I mean, I invite you to watch this ISIT lecture: <a href="https://www.youtube.com/watch?v=O_uBxFGk-U4&amp;t=1563s" rel="nofollow noreferrer">The Spirit of Information Theory</a> by Prof David TSe.</p>
<p>Check also the paper <a href="https://monoskop.org/images/2/2f/Shannon_Claude_E_1956_The_Bandwagon.pdf" rel="nofollow noreferrer">Bandwagon</a> by Claude Channon himself explaining when information theory might be useful and when it is not appropriate for use.</p>
<p>This <a href="https://arxiv.org/abs/1802.05968" rel="nofollow noreferrer">paper</a> helps you get you started and for comprehensive details read <a href="https://rads.stackoverflow.com/amzn/click/com/0471062596" rel="nofollow noreferrer" rel="nofollow noreferrer">Elements of Information theory</a>.</p>
Practical way of explaining "Information Theory"
<p>Information theory comes into play where ever encoding &amp; decoding is present. For example: compression(multimedia), cryptography.</p>

<p>In Information Theory we encounter terms like "Entropy", "Self Information", "Mutual Information" and entire subject is based on these terms. Which just sound nothing more than abstract. Frankly, they don't really make any sense.</p>

<p>Is there any book/material/explanation (if you can) which explains these things in a practical way? </p>

<p><strong>EDIT:</strong></p>

<blockquote>
  <p><a href="http://amzn.com/0486240614" rel="noreferrer">An Introduction to Information Theory: symbols, signals &amp; noise by John Robinson Pierce</a> is <strong>The Book</strong> that explains it the way I want (practically). Its too good. I started reading it.</p>
</blockquote>