<p>This is my first answer at SO, so here it goes..</p>

<p>Your understanding in both parts seem to be correct, according to this paper :</p>

<p><a href="http://arxiv.org/abs/1411.2738" rel="noreferrer">http://arxiv.org/abs/1411.2738</a></p>

<p>The paper explains word2vec in detail and at the same time, keeps it very simple - it's worth a read for a thorough understanding of the neural net architecture used in word2vec.</p>

<ul>
<li>The structure of Skip Gram does use a single neural net, with input as one-hot encoded target-word and <strong>expected-output</strong> as one-hot encoded context words. After the neural-net is trained on the text-corpus, the input weight matrix <strong>W</strong>   is used as the input-vector representations of words in the corpus and the output weight matrix <strong>W'</strong> which is shared across all the <strong>C</strong> outputs (output-vectors in the terminology of the question, but avoiding that to prevent confusion with output-vector representations used next..), becomes the output-vector representations of words. Usually the output-vector representations are ignored, and the input-vector representations, <strong>W</strong> are used as the word embeddings. To get into the dimensionality of the matrices, if we assume a vocabulary size of <strong>V</strong>, size of hidden layer as <strong>N</strong>, we will have <strong>W</strong> as <strong>(V,N)</strong> matrix, with each row representing the input vector of the indexed word in the vocabulary. <strong>W'</strong> will be a <strong>(N,V)</strong> matrix, with each column representing the output vector of the indexed word. In this way we get N-dimensional vectors for words.</li>
<li>As you mentioned, each of the outputs(avoiding using the term output vector) is of size <strong>V</strong> and are the result of a softmax function, with each node in the output giving the probability of the word occurring as a context word for the given target word, resulting in the outputs not being one-hot encoded.But the expected outputs are indeed one-hot encoded, i.e in training phase, the error is computed by subtracting the one-hot encoded vector of the actual word occurring at that context position, from the neural-net output and then the weights are updated using gradient descent.</li>
</ul>

<p>Referring to the example you mentioned, with <strong>C</strong>=1 and with a vocabulary of  ['quick', 'fox', 'jumped', 'lazy', 'dog'] </p>

<p>If the output from the skip-gram is [0.2 0.6 0.01 0.1 0.09], where the correct target word is 'fox' then error is calculated as -</p>

<p>[0 1 0 0 0] - [0.2 0.6 0.01 0.1 0.09] = [-0.2 0.4 -0.01 -0.1 -0.09]</p>

<p>and the weight matrices are updated to minimize this error.</p>

<p>Hope this helps !</p>
Understanding Word2Vec's Skip-Gram Structure and Output
<p>My question is two-fold, but hopefully not too complicated. And both parts specifically pertain to the Skip-Gram model in Word2Vec:</p>

<ul>
<li><p>The first part is about structure: as far as I understand it, the Skip-Gram model is based on one neural network with one input weight matrix <strong>W</strong>, one hidden layer of size N, and C output weight matrices <strong>W'</strong> each used to produce one of the C output vectors. Is this correct?</p></li>
<li><p>The second part is about the output vectors: as far as I understand it, each output vector is of size V and is a result of a Softmax function. Each output vector <em>node</em> corresponds to the index of a word in the vocabulary, and the value of each node is the probability that the corresponding word occurs at that context location (for a given input word). The target output vectors are not, however, one-hot encoded, even if the training instances are. Is this correct?</p></li>
</ul>

<p>The way I imagine it is something along the following lines (made-up example):</p>

<p>Assuming the vocabulary ['quick', 'fox', 'jumped', 'lazy', 'dog'] and a context of C=1, and assuming that for the input word 'jumped' I see the two output vectors looking like this:</p>

<p>[0.2 <strong>0.6</strong> 0.01 0.1 0.09]</p>

<p>[0.2 0.2 0.01 0.16 <strong>0.43</strong>]</p>

<p>I would interpret this as 'fox' being the most likely word to show up before 'jumped' (p=0.6), and 'dog' being the most likely to show up after it (p=0.43).</p>

<p>Do I have this right? Or am I completely off? Any help is appreciated.</p>