<p>I think you got right way. What you need is just a lexicon to determine whether a given word is a known word or unknown word. <a href="http://rdrpostagger.sourceforge.net/" rel="nofollow">RDRPOSTagger</a> provides a piece of code to compute tagging accuracies for known words and unknown words. See the function <code>computeAccuracies(lexicon, goldCorpus, taggedCorpus)</code> in the <code>Eval.py</code> module in the <code>Utility</code> package. </p>

<p>You might want to look at <a href="http://arxiv.org/abs/1412.4021" rel="nofollow">this paper</a> which presents tagging results (for known words and unknown words) of 3 POS and morphological taggers on 13 languages including Bulgarian, Czech, Dutch, English, French, German, Hindi, Italian, Portuguese, Spanish, Swedish, Thai and Vietnamese.</p>
POS accuracy of known and unknown words
<p>How do I calculate the accuracy of known and unknown words in part of speech tagging? For example for known words, is it dividing the correctly tagged known words by all the known words ? Any other ways ?</p>