title: Latest Pre-trained Multilingual Word Embedding
body: <p>Are there any latest <strong>pre-trained multilingual word embeddings</strong> (multiple languages are jointly mapped to a same vector space)?</p>
  
  <p>I have looked at the following but they don't fit my needs:</p>
  
  <ol>
  <li>FastText / MUSE (<a href="https://fasttext.cc/docs/en/aligned-vectors.html" rel="noreferrer">https://fasttext.cc/docs/en/aligned-vectors.html</a>): this one seems too old, and the word vectors are not using subwords / wordpiece information.</li>
  <li>LASER (<a href="https://github.com/yannvgn/laserembeddings" rel="noreferrer">https://github.com/yannvgn/laserembeddings</a>): I'm now using this one, it's using subword information (via BPE), however, it's suggested that not to use this for word embedding because it's designed to embed sentences (<a href="https://github.com/facebookresearch/LASER/issues/69" rel="noreferrer">https://github.com/facebookresearch/LASER/issues/69</a>).</li>
  <li>BERT multilingual (bert-base-multilingual-uncased in <a href="https://huggingface.co/transformers/pretrained_models.html" rel="noreferrer">https://huggingface.co/transformers/pretrained_models.html</a>): it's contextualised embeddings that can be used to embed sentences, and seems not good at embedding words without contexts.</li>
  </ol>
  
  <p>Here is the problem I'm trying to solve:</p>
  
  <p>I have a list of company names, which can be in any language (mainly English), and I have a list of keywords in English to measure how close a given company name is with regards to the keywords. Now I have a simple keyword matching solution, but I want to improve it using pretrained embeddings. As you can see in the following examples, there are several challenges:</p>
  
  <ol>
  <li>keyword and brand name is not separated by space (now I'm using package "wordsegment" to split words into subwords), so embedding with subword info should help a lot</li>
  <li>keyword list is not extensive and company name could be in different languages (that's why I want to use embedding, because "soccer" is close to "football")</li>
  </ol>
  
  <p>Examples of company names: "cheapfootball ltd.", "wholesalefootball ltd.", "footballer ltd.", "soccershop ltd."</p>
  
  <p>Examples of keywords: "football"</p>