<p>I've been wondering about this too. It's not really clear to me what they're doing, but this is what I found.</p>

<p>In the <a href="http://arxiv.org/pdf/1606.07792v1.pdf" rel="noreferrer">paper on wide and deep learning</a>, they describe the embedding vectors as being randomly initialized and then adjusted during training to minimize error.</p>

<p>Normally when you do embeddings, you take some arbitrary vector representation of the data (such as one-hot vectors) and then multiply it by a matrix that represents the embedding. This matrix can be found by PCA or while training by something like t-SNE or word2vec.</p>

<p>The actual code for the embedding_column is <a href="https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/layers/python/layers/feature_column.py" rel="noreferrer">here</a>, and it's implemented as a class called _EmbeddingColumn which is a subclass of _FeatureColumn. It stores the embedding matrix inside its sparse_id_column attribute. Then, the method to_dnn_input_layer applies this embedding matrix to produce the embeddings for the next layer.</p>

<pre><code> def to_dnn_input_layer(self,
                         input_tensor,
                         weight_collections=None,
                         trainable=True):
    output, embedding_weights = _create_embedding_lookup(
        input_tensor=self.sparse_id_column.id_tensor(input_tensor),
        weight_tensor=self.sparse_id_column.weight_tensor(input_tensor),
        vocab_size=self.length,
        dimension=self.dimension,
        weight_collections=_add_variable_collection(weight_collections),
        initializer=self.initializer,
        combiner=self.combiner,
        trainable=trainable)
</code></pre>

<p>So as far as I can see, it seems like the embeddings are formed by applying whatever learning rule you're using (gradient descent, etc.) to the embedding matrix.</p>
tf.contrib.layers.embedding_column from tensor flow
<p>I am going through tensorflow tutorial <a href="https://www.tensorflow.org/versions/r0.10/tutorials/wide_and_deep/index.html#tensorflow-wide-deep-learning-tutorial" rel="noreferrer">tensorflow</a>. I would like to find description of the following line:</p>

<pre><code>tf.contrib.layers.embedding_column
</code></pre>

<p>I wonder if it uses word2vec or anything else, or maybe I am thinking in completely wrong direction. I tried to click around on GibHub, but found nothing. I am guessing looking on GitHub is not going to be easy, since python might refer to some C++ libraries. Could anybody point me in the right direction?</p>