<p>This is not an easy problem.</p>

<p>You suggest replacing the words on the data preprocessing level. To do so, you would need word alignment that would tell you what source words match your target word. There are tools for that like <a href="https://github.com/clab/fast_align" rel="nofollow noreferrer">FastAlign</a>. Even when you have the alignment, there is no guarantee that the copied source word will be in the target vocabulary.</p>

<p>Some people tried to solve this issue on the modeling level and include explicit copy mechanisms in their network (like <a href="https://arxiv.org/abs/1603.06393" rel="nofollow noreferrer">this paper</a>), however it makes the network rather complicated and gives only a little improvement.</p>

<p>The most common workaround for this issue is using subword-based vocabulary like <a href="https://github.com/glample/fastBPE" rel="nofollow noreferrer">BPE</a> or <a href="https://github.com/google/sentencepiece" rel="nofollow noreferrer">SentencePiece</a>. With these methods, unfrequent words get segmented into smaller units, so nothing is in the end out of vocabulary. If the word is the same both on the source and target side (this often happens with proper names), it will get segmented in the same way on the source and target side and the model will learn that copying the word fragments is what it usually should do.</p>
Replacing UNKNOWN words with the sorce word in MT
<p>I'm working on  Machine learning AI translation system, and I want to make it more adaptable my code now when the word is new will place <em>UNK</em> which stands for <em>UNKNOWN</em> and leave it, but I want to copy the same word and past it back instead of printing <em>UNK</em>, so if a new word comes it should pass back the same word as translation instead of <em>UNK</em> my code looks like this for now:</p>

<p>any ideas what shall I change : </p>

<pre><code># Adding the word 'UNK' to the end of the array (stands for UNKNOWN words)
    X_ix_to_word.append('UNK')

    # Creating the word-to-index dictionary from the array created above
    X_word_to_ix = {word:ix for ix, word in enumerate(X_ix_to_word)}

    # Converting each word to its index value
    for i, sentence in enumerate(X):
        for j, word in enumerate(sentence):
            if word in X_word_to_ix:
                X[i][j] = X_word_to_ix[word]
            else:
                X[i][j] = X_word_to_ix['UNK']

    y_ix_to_word = [word[0] for word in y_vocab]
    y_ix_to_word.insert(0, 'ZERO')
    y_ix_to_word.append('UNK')
    y_word_to_ix = {word:ix for ix, word in enumerate(y_ix_to_word)}
    for i, sentence in enumerate(y):
        for j, word in enumerate(sentence):
            if word in y_word_to_ix:
                y[i][j] = y_word_to_ix[word]
            else:
                y[i][j] = y_word_to_ix['UNK']
    return (X, len(X_vocab)+2, X_word_to_ix, X_ix_to_word, y, len(y_vocab)+2, y_word_to_ix, y_ix_to_word)

def load_test_data(source, X_word_to_ix, max_len):
    f = open(source, 'r')
    X_data = f.read()
    f.close()

    X = [text_to_word_sequence(x)[::-1] for x in X_data.split('\n') if len(x) &gt; 0 and len(x) &lt;= max_len]
    for i, sentence in enumerate(X):
        for j, word in enumerate(sentence):
            if word in X_word_to_ix:
                X[i][j] = X_word_to_ix[word]
            else:
                X[i][j] = X_word_to_ix['UNK']
    return X

</code></pre>