{"action":"created","issue":{"url":"https://api.github.com/repos/shangjingbo1226/AutoPhrase/issues/50","repository_url":"https://api.github.com/repos/shangjingbo1226/AutoPhrase","labels_url":"https://api.github.com/repos/shangjingbo1226/AutoPhrase/issues/50/labels{/name}","comments_url":"https://api.github.com/repos/shangjingbo1226/AutoPhrase/issues/50/comments","events_url":"https://api.github.com/repos/shangjingbo1226/AutoPhrase/issues/50/events","html_url":"https://github.com/shangjingbo1226/AutoPhrase/issues/50","id":385589030,"node_id":"MDU6SXNzdWUzODU1ODkwMzA=","number":50,"title":"[Fatal Error]: Buffer Limit Exceeded Error! You may want to modify the buffer limit in the Tokenizer.java","user":{"login":"mohammadzainabbas","id":19189095,"node_id":"MDQ6VXNlcjE5MTg5MDk1","avatar_url":"https://avatars3.githubusercontent.com/u/19189095?v=4","gravatar_id":"","url":"https://api.github.com/users/mohammadzainabbas","html_url":"https://github.com/mohammadzainabbas","followers_url":"https://api.github.com/users/mohammadzainabbas/followers","following_url":"https://api.github.com/users/mohammadzainabbas/following{/other_user}","gists_url":"https://api.github.com/users/mohammadzainabbas/gists{/gist_id}","starred_url":"https://api.github.com/users/mohammadzainabbas/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mohammadzainabbas/subscriptions","organizations_url":"https://api.github.com/users/mohammadzainabbas/orgs","repos_url":"https://api.github.com/users/mohammadzainabbas/repos","events_url":"https://api.github.com/users/mohammadzainabbas/events{/privacy}","received_events_url":"https://api.github.com/users/mohammadzainabbas/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":5,"created_at":"2018-11-29T06:30:51Z","updated_at":"2018-12-11T08:32:57Z","closed_at":null,"author_association":"NONE","body":"This issue occurred when I ran `./phrasal_segmentation.sh` and at `Generating Output` stage. \r\n\r\n![image](https://user-images.githubusercontent.com/19189095/49203393-eaea4d80-f3c9-11e8-9e37-19f209f1cb0d.png)\r\n\r\nI have used my **own data corpus** (_of size 22 MB_), while still using the _pre-trained model_. (As mentioned in Issue #22, I have changed the `text_to_seg` in the `phrasal_segmentation.sh` script.)\r\n\r\nI have tried to increase the Buffer limit from 8192 `default size` to 16384 (`2 x 8192`), 32768 (`4 x 8192`), 65536 (`8 x 8192`), 131072 (`16 x 8192`), 262144 (`32 x 8192`) but no use.\r\n\r\nHow may I resolve this issue ?"},"comment":{"url":"https://api.github.com/repos/shangjingbo1226/AutoPhrase/issues/comments/446115383","html_url":"https://github.com/shangjingbo1226/AutoPhrase/issues/50#issuecomment-446115383","issue_url":"https://api.github.com/repos/shangjingbo1226/AutoPhrase/issues/50","id":446115383,"node_id":"MDEyOklzc3VlQ29tbWVudDQ0NjExNTM4Mw==","user":{"login":"mohammadzainabbas","id":19189095,"node_id":"MDQ6VXNlcjE5MTg5MDk1","avatar_url":"https://avatars3.githubusercontent.com/u/19189095?v=4","gravatar_id":"","url":"https://api.github.com/users/mohammadzainabbas","html_url":"https://github.com/mohammadzainabbas","followers_url":"https://api.github.com/users/mohammadzainabbas/followers","following_url":"https://api.github.com/users/mohammadzainabbas/following{/other_user}","gists_url":"https://api.github.com/users/mohammadzainabbas/gists{/gist_id}","starred_url":"https://api.github.com/users/mohammadzainabbas/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mohammadzainabbas/subscriptions","organizations_url":"https://api.github.com/users/mohammadzainabbas/orgs","repos_url":"https://api.github.com/users/mohammadzainabbas/repos","events_url":"https://api.github.com/users/mohammadzainabbas/events{/privacy}","received_events_url":"https://api.github.com/users/mohammadzainabbas/received_events","type":"User","site_admin":false},"created_at":"2018-12-11T08:32:57Z","updated_at":"2018-12-11T08:32:57Z","author_association":"NONE","body":"@yanhongfirst I have tried your fix on my corpus, but no use, getting the same issue.\r\n@shangjingbo1226 I have tried removing all special characters too with `re.sub('[^A-Za-z0-9]+', ' ', mystring)` as mentioned [here](https://stackoverflow.com/a/5843547/6390175), and ran it again. But no use. "}}
{"actor":{"display_login":"mohammadzainabbas"}}