<p>I am surprised the previous answers haven't mentioned word embedding. Word embedding algorithm can produce word vectors for each word a given dataset. These algorithms can nfer word vectors from  the context. For instance,  by looking at the context of the following sentences we can say that "clever" and "smart" is somehow related. Because the context is almost the same. <br/></p>

<p><code>
He is a clever guy
He is a smart guy 
</code></p>

<p>A co-occurrence matrix can be constructed to do this. However, it is too inefficient. A famous technique designed for this purpose is called Word2Vec. It can be studied from the following papers.<br/> 
<a href="https://arxiv.org/pdf/1411.2738.pdf" rel="nofollow noreferrer">https://arxiv.org/pdf/1411.2738.pdf</a>  <br/>
<a href="https://arxiv.org/pdf/1402.3722.pdf" rel="nofollow noreferrer">https://arxiv.org/pdf/1402.3722.pdf</a>  <br/></p>

<p>I have been using it for Swedish. It is quite effective in detecting similar words and completely unsupervised. </p>

<p>A package could be find in gensim and tensorflow. </p>
Feature extraction from a single word
<p>Usually one wants to get a feature from a text by using the bag of words approach, counting the words and calculate different measures, for example tf-idf values, like this:  <a href="https://stackoverflow.com/questions/4207057/how-to-include-words-as-numerical-feature-in-classification">How to include words as numerical feature in classification</a></p>

<p>But my problem is different, I want to extract a feature vector from a single word. I want to know for example that potatoes and french fries are close to each other in the vector space, since they are both made of potatoes. I want to know that milk and cream also are close, hot and warm, stone and hard and so on.</p>

<p>What is this problem called? Can I learn the similarities and features of words by just looking at a large number documents?</p>

<p>I will not make the implementation in English, so I can't use databases.</p>