{"action":"created","issue":{"url":"https://api.github.com/repos/Embedding/Chinese-Word-Vectors/issues/2","repository_url":"https://api.github.com/repos/Embedding/Chinese-Word-Vectors","labels_url":"https://api.github.com/repos/Embedding/Chinese-Word-Vectors/issues/2/labels{/name}","comments_url":"https://api.github.com/repos/Embedding/Chinese-Word-Vectors/issues/2/comments","events_url":"https://api.github.com/repos/Embedding/Chinese-Word-Vectors/issues/2/events","html_url":"https://github.com/Embedding/Chinese-Word-Vectors/issues/2","id":322472849,"number":2,"title":"Vocabulary size和embedding中的词汇量不符","user":{"login":"cai-lw","id":9162726,"avatar_url":"https://avatars2.githubusercontent.com/u/9162726?v=4","gravatar_id":"","url":"https://api.github.com/users/cai-lw","html_url":"https://github.com/cai-lw","followers_url":"https://api.github.com/users/cai-lw/followers","following_url":"https://api.github.com/users/cai-lw/following{/other_user}","gists_url":"https://api.github.com/users/cai-lw/gists{/gist_id}","starred_url":"https://api.github.com/users/cai-lw/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cai-lw/subscriptions","organizations_url":"https://api.github.com/users/cai-lw/orgs","repos_url":"https://api.github.com/users/cai-lw/repos","events_url":"https://api.github.com/users/cai-lw/events{/privacy}","received_events_url":"https://api.github.com/users/cai-lw/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":3,"created_at":"2018-05-12T01:41:06Z","updated_at":"2018-05-12T14:06:27Z","closed_at":"2018-05-12T03:54:25Z","author_association":"NONE","body":"我下载了wikipedia语料的word+char模型，用gensim载入之后显示词汇量为352281，但README中的vocabulary size写的是2129K\r\n按我的理解，两者应该是一样的。请问是我的理解有误还是数据有误？"},"comment":{"url":"https://api.github.com/repos/Embedding/Chinese-Word-Vectors/issues/comments/388557614","html_url":"https://github.com/Embedding/Chinese-Word-Vectors/issues/2#issuecomment-388557614","issue_url":"https://api.github.com/repos/Embedding/Chinese-Word-Vectors/issues/2","id":388557614,"user":{"login":"shenshen-hungry","id":30199885,"avatar_url":"https://avatars0.githubusercontent.com/u/30199885?v=4","gravatar_id":"","url":"https://api.github.com/users/shenshen-hungry","html_url":"https://github.com/shenshen-hungry","followers_url":"https://api.github.com/users/shenshen-hungry/followers","following_url":"https://api.github.com/users/shenshen-hungry/following{/other_user}","gists_url":"https://api.github.com/users/shenshen-hungry/gists{/gist_id}","starred_url":"https://api.github.com/users/shenshen-hungry/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/shenshen-hungry/subscriptions","organizations_url":"https://api.github.com/users/shenshen-hungry/orgs","repos_url":"https://api.github.com/users/shenshen-hungry/repos","events_url":"https://api.github.com/users/shenshen-hungry/events{/privacy}","received_events_url":"https://api.github.com/users/shenshen-hungry/received_events","type":"User","site_admin":false},"created_at":"2018-05-12T14:06:27Z","updated_at":"2018-05-12T14:06:27Z","author_association":"COLLABORATOR","body":"> @shenshen-hungry 但是这个表格后面有这么一句话\r\n> \r\n> All words are concerned, including low frequency words.\r\n\r\n这里是说在统计corpus信息的时候是针对整个corpus的，也就是corpus里面所有的词汇都被统计了。在训练词向量的时候，通常做法是去掉低频词，所以词向量的词表要小于corpus的词表。\r\n\r\n论文稍后会上传arxiv供引用，需要稍等一下下。"}}
{"actor":{"display_login":"shenshen-hungry"}}