<p>You might be interested in <a href="https://arxiv.org/pdf/1801.07779.pdf" rel="nofollow noreferrer">The WiLI benchmark dataset for written language identification</a>. The high level-answer you can also find in the paper is the following:</p>

<ul>
<li>Clean the text: Remove things you don't want / need; make unicode un-ambiguious by applying a normal form.</li>
<li>Feature Extraction: Count n-grams, create tf-idf features. Something like that</li>
<li>Train a classifier on the features: Neural networks, SVMs, Naive Bayes, ... whatever you think could work.</li>
</ul>
How does language detection work?
<p>I have been wondering for some time how does Google translate(or maybe a hypothetical translator) detect language from the string entered in the "from" field. I have been thinking about this and only thing I can think of is looking for words that are unique to a language in the input string. The other way could be to check sentence formation or other semantics in addition to keywords. But this seems to be a very difficult task considering different languages and their semantics. I did some research to find that there are ways that use n-gram sequences and use some statistical models to detect language. Would appreciate a high level answer too.</p>