<p>Depends on your training data.</p>

<p>The embeddings are learned when you train the model. If the examples containing "dog" and "doghouse" are very similar I would expect the embeddings would be close. The embeddings are close when the semantics within the training data are close.</p>

<p>If you train a model to distringuish between animals and other objects (for example) then the two words should be fairly far away.</p>

<p>For the second part, given that you have reasonable training examples that reflect the relation you want over the strings you should get embeddings that reflect that relationship. Check out <a href="https://arxiv.org/pdf/1310.4546.pdf" rel="nofollow noreferrer">this paper</a> where they train embedding to reflect many relationships between the words (for example capital to country).</p>
how embedding columns work to assign numbers to embeddings vectors?
<p>I have a project where I use data where we have numeric features and String features to make a binary classifier. And I was reading the following explanation about features columns in tensorflow :<a href="https://www.tensorflow.org/get_started/feature_columns" rel="nofollow noreferrer">https://www.tensorflow.org/get_started/feature_columns</a>.</p>

<p>I have a problem to understand exactly what embedding columns will do to String features:
in the tensorflow example, we have a feature whose values are in ["dog","spoon","scissors","guitar"]. We use a categorical column to convert the Strings to integers which are index to a lookup table where each String is finally mapped to a vector of low dimensions (initialized with random float numbers). It is said that the assignments of the embeddings vectors happen during training and that embedding columns increase our model's capabilities, since an embeddings vector learns new relationships between categories from the training data.</p>

<p><a href="https://i.stack.imgur.com/j9GLp.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/j9GLp.jpg" alt="embedding columns in tensorflow"></a></p>

<p>My question is : do embedding columns manage to create vectors that represent the similarity between the Strings as "the vector learns new relationships between categories" ? For instance, if we add "doghouse" to the above example vocabulary, the distance between the embeddings vectors of "dog" and "doghouse" will be shorter than with the vectors of the other words.</p>

<p>if I push the question further, we can have a String categorical feature whose values are in ["red circle","red square","blue circle","blue square"], and the question will be: will the embedding vectors express the relationship between categories with the similarity with the color and the shape?</p>

<p>Thanks in advance for your help.</p>