<p>Had you read the source code of any decoder like <a href="https://github.com/tensorflow/tensorflow/blob/r1.1/tensorflow/contrib/legacy_seq2seq/python/ops/seq2seq.py#L648" rel="nofollow noreferrer">this one</a> you would get to know that it represents the number of attentions. </p>

<p>Sometimes there are several attentions(hierarchical attentions), for instance this one(as depicted bellow) in <a href="http://arxiv.org/abs/1602.06023" rel="nofollow noreferrer">this paper</a>.<br>
TL;DR; the first one is for the word and the second one is for the sentence.<br>
 Please check this graph: 
<a href="https://i.stack.imgur.com/aXPni.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/aXPni.png" alt="enter image description here"></a></p>
"attention heads" in seq2seq.embedding_attention_seq2seq of tensorflow
<p>I'm new to the tensorflow and trying to implement the "seq2seq" model according to the tutorial. I'm not sure about one argument "num_heads" (default=1) of the func "embedding_attention_seq2seq". What does it represent? I didn't find it in the related papers.</p>