{"action":"created","issue":{"url":"https://api.github.com/repos/zhangsonglei/Ngram/issues/1","repository_url":"https://api.github.com/repos/zhangsonglei/Ngram","labels_url":"https://api.github.com/repos/zhangsonglei/Ngram/issues/1/labels{/name}","comments_url":"https://api.github.com/repos/zhangsonglei/Ngram/issues/1/comments","events_url":"https://api.github.com/repos/zhangsonglei/Ngram/issues/1/events","html_url":"https://github.com/zhangsonglei/Ngram/issues/1","id":330512104,"node_id":"MDU6SXNzdWUzMzA1MTIxMDQ=","number":1,"title":"java代码的训练结果与ngram-count结果不一致的问题","user":{"login":"tiandiweizun","id":13550295,"node_id":"MDQ6VXNlcjEzNTUwMjk1","avatar_url":"https://avatars0.githubusercontent.com/u/13550295?v=4","gravatar_id":"","url":"https://api.github.com/users/tiandiweizun","html_url":"https://github.com/tiandiweizun","followers_url":"https://api.github.com/users/tiandiweizun/followers","following_url":"https://api.github.com/users/tiandiweizun/following{/other_user}","gists_url":"https://api.github.com/users/tiandiweizun/gists{/gist_id}","starred_url":"https://api.github.com/users/tiandiweizun/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tiandiweizun/subscriptions","organizations_url":"https://api.github.com/users/tiandiweizun/orgs","repos_url":"https://api.github.com/users/tiandiweizun/repos","events_url":"https://api.github.com/users/tiandiweizun/events{/privacy}","received_events_url":"https://api.github.com/users/tiandiweizun/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2018-06-08T04:03:50Z","updated_at":"2018-07-03T06:42:35Z","closed_at":"2018-07-03T06:42:21Z","author_association":"NONE","body":"ngram-count训练参数：\r\n1.统计词频\r\nD:\\cygwin64\\home\\chubb\\srilm\\bin\\cygwin64\\ngram-\r\ncount.exe -text .\\ngram_data -order 3 -write .\\ngram_data_CN\r\n2.语言模型\r\nD:\\cygwin64\\home\\chubb\\srilm\\bin\\cygwin64\\ngram-\r\ncount.exe -read .\\ngram_data_CN -order 3 -lm .\\ngram_data_LM -ukndiscount1\r\n\r\njava代码训练参数\r\njava -cp ChineseSpellingCheck.jar hust.tools.ngram.app.NGramLMTrain train.dat utf-8 3 Y kn trigram.lm text\r\n\r\n试验结果：\r\nngram-count结果\r\n\\data\\\r\nngram 1=5685\r\nngram 2=531648\r\nngram 3=1224327\r\n\r\n\\1-grams:\r\n-2.663042\t!\t-1.557407\r\n-2.955509\t\"\t-0.7174896\r\n-3.874366\t#\t-1.386763\r\n-4.771382\t$\t-0.4257379\r\n-3.893115\t%\t-1.282743\r\n-3.387168\t&\t-1.997125\r\n-3.668719\t'\t-0.6878299\r\n-2.921485\t(\t-0.7731137\r\n-2.906739\t)\t-1.016507\r\n-3.555362\t*\t-1.446827\r\n\r\njava代码结果\r\n3\r\n5691\r\n531679\r\n1224330\r\n-4.596007237193767\t\t-0.30495468198354575\r\n-2.6632636494389934\t!\t-0.6929173857907096\r\n-2.9559188903417892\t\"\t-0.46791485367975977\r\n-3.877592719528297\t#\t-0.5462744250640476\r\n-4.79732935876629\t$\t-0.34568711194645774\r\n-3.8964839459321765\t%\t-0.4328627851501955\r\n-3.388233134376914\t&\t-1.3182228221695054\r\n-3.670735836633295\t'\t-0.42412574478336396\r\n-2.9218657541928734\t(\t-0.4112865266073557\r\n-2.90710789326123\t)\t-0.5585262636460637\r\n-3.5569207307321604\t*\t-0.5499648255080977\r\n-3.1974033687466386\t+\t-0.5313955482036398\r\n\r\n**问题：\r\n1.概率基本一致，但是回退概率不一致\r\n2.数量有微小差别，除了加入了<unk>，还有第一个概率较高的没有显示，还有最后几个 unigram最后的4个？，估计是特殊字符，没有显示，所以比ngarm-count的1-gram多了6个。**\r\n"},"comment":{"url":"https://api.github.com/repos/zhangsonglei/Ngram/issues/comments/402029944","html_url":"https://github.com/zhangsonglei/Ngram/issues/1#issuecomment-402029944","issue_url":"https://api.github.com/repos/zhangsonglei/Ngram/issues/1","id":402029944,"node_id":"MDEyOklzc3VlQ29tbWVudDQwMjAyOTk0NA==","user":{"login":"tiandiweizun","id":13550295,"node_id":"MDQ6VXNlcjEzNTUwMjk1","avatar_url":"https://avatars0.githubusercontent.com/u/13550295?v=4","gravatar_id":"","url":"https://api.github.com/users/tiandiweizun","html_url":"https://github.com/tiandiweizun","followers_url":"https://api.github.com/users/tiandiweizun/followers","following_url":"https://api.github.com/users/tiandiweizun/following{/other_user}","gists_url":"https://api.github.com/users/tiandiweizun/gists{/gist_id}","starred_url":"https://api.github.com/users/tiandiweizun/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tiandiweizun/subscriptions","organizations_url":"https://api.github.com/users/tiandiweizun/orgs","repos_url":"https://api.github.com/users/tiandiweizun/repos","events_url":"https://api.github.com/users/tiandiweizun/events{/privacy}","received_events_url":"https://api.github.com/users/tiandiweizun/received_events","type":"User","site_admin":false},"created_at":"2018-07-03T06:42:35Z","updated_at":"2018-07-03T06:42:35Z","author_association":"NONE","body":"谢谢"}}
{"actor":{"display_login":"tiandiweizun"}}