<p><strong>TL;DR</strong></p>

<p>The opposite is actually the case. Higher precision calculations are less desired by frameworks like TensorFlow. This is due to slower training and larger models (more ram and disc space).</p>

<p><strong>The long version</strong></p>

<p>Neural networks actually benefit from using lower precision representations. <a href="https://arxiv.org/pdf/1502.02551.pdf" rel="nofollow noreferrer">This paper</a> is a good introduction to the topic.</p>

<blockquote>
  <p>The key finding of our exploration is that deep neural networks  can 
  be  trained  using  low-precision  fixed-point arithmetic, provided
  that the stochastic rounding scheme is applied while operating on
  fixed-point numbers.</p>
</blockquote>

<p>They use 16 bit fixed point number rather than the much higher precession 32 bit floating point number (more information on their difference <a href="https://stackoverflow.com/questions/7524838/fixed-point-vs-floating-point-number">here</a>).</p>

<p>The following image was taken from that paper. It shows the test error for different rounding schemes as well as the number of bits dedicated to the integer part of the fixed point representation. As you can see the solid red and blue lines (16 bit fixed) have a very similar error to the black line (32 bit float).</p>

<p><a href="https://i.stack.imgur.com/RMGoS.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/RMGoS.png" alt="enter image description here"></a></p>

<p>The main benefit/driver for going to a lower precision is computational cost and storage of weights. So the higher precision hardware would not give enough of an accuracy increase to out way the cost of slower computation.</p>

<p>Studies like this I believe are a large driver behind the specs for neural network specific processing hardware, such as <a href="https://en.wikipedia.org/wiki/Tensor_processing_unit" rel="nofollow noreferrer">Google's new TPU</a>. Even though most GPUs don't support 16 bit floats yet Google is <a href="https://github.com/tensorflow/tensorflow/issues/1300" rel="nofollow noreferrer">working to support it</a>.</p>
Leveraging high precision gpu's in tensor flow
<p>Hi I was reading the <a href="https://www.tensorflow.org/programmers_guide/using_gpu" rel="noreferrer">using GPUs page</a> at tensor flow and I was wondering if gpu precision performance was ever a factor in tensor flow. For example given a machine with two cards,</p>

<blockquote>
  <p>gaming gpu</p>
</blockquote>

<p>+</p>

<blockquote>
  <p>workstation gpu</p>
</blockquote>

<p>is there any implementation that would provide the workstation card's higher precision performance could overcome the slower clock speed?</p>

<p>I'm not sure if these situations would exist in the context of gradient decent or network performance after training or elsewhere entirely but I would love to get some more information on the topic!</p>

<p>Thanks in advance.</p>