{"action":"created","issue":{"url":"https://api.github.com/repos/aoldoni/tetre/issues/34","repository_url":"https://api.github.com/repos/aoldoni/tetre","labels_url":"https://api.github.com/repos/aoldoni/tetre/issues/34/labels{/name}","comments_url":"https://api.github.com/repos/aoldoni/tetre/issues/34/comments","events_url":"https://api.github.com/repos/aoldoni/tetre/issues/34/events","html_url":"https://github.com/aoldoni/tetre/issues/34","id":295011247,"number":34,"title":"process killed ","user":{"login":"timbilino","id":12633454,"avatar_url":"https://avatars3.githubusercontent.com/u/12633454?v=4","gravatar_id":"","url":"https://api.github.com/users/timbilino","html_url":"https://github.com/timbilino","followers_url":"https://api.github.com/users/timbilino/followers","following_url":"https://api.github.com/users/timbilino/following{/other_user}","gists_url":"https://api.github.com/users/timbilino/gists{/gist_id}","starred_url":"https://api.github.com/users/timbilino/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/timbilino/subscriptions","organizations_url":"https://api.github.com/users/timbilino/orgs","repos_url":"https://api.github.com/users/timbilino/repos","events_url":"https://api.github.com/users/timbilino/events{/privacy}","received_events_url":"https://api.github.com/users/timbilino/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-02-07T05:26:26Z","updated_at":"2018-02-09T10:23:33Z","closed_at":null,"author_association":"NONE","body":"Hello, i try to run the commad:\r\n./bin/tetre extract --tetre_word improves --tetre_output html\r\nbut, the process is killed. I use Ubuntu and python 3.5\r\n\r\n(.env) ubuntu@ip-172-31-22-210:~/.local/lib/python3.5/tetre$ ./bin/tetre extract --tetre_word improves --tetre_output html\r\nKilled\r\n(.env) ubuntu@ip-172-31-22-210:~/.local/lib/python3.5/tetre$\r\n"},"comment":{"url":"https://api.github.com/repos/aoldoni/tetre/issues/comments/364394882","html_url":"https://github.com/aoldoni/tetre/issues/34#issuecomment-364394882","issue_url":"https://api.github.com/repos/aoldoni/tetre/issues/34","id":364394882,"user":{"login":"aoldoni","id":1471238,"avatar_url":"https://avatars0.githubusercontent.com/u/1471238?v=4","gravatar_id":"","url":"https://api.github.com/users/aoldoni","html_url":"https://github.com/aoldoni","followers_url":"https://api.github.com/users/aoldoni/followers","following_url":"https://api.github.com/users/aoldoni/following{/other_user}","gists_url":"https://api.github.com/users/aoldoni/gists{/gist_id}","starred_url":"https://api.github.com/users/aoldoni/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/aoldoni/subscriptions","organizations_url":"https://api.github.com/users/aoldoni/orgs","repos_url":"https://api.github.com/users/aoldoni/repos","events_url":"https://api.github.com/users/aoldoni/events{/privacy}","received_events_url":"https://api.github.com/users/aoldoni/received_events","type":"User","site_admin":false},"created_at":"2018-02-09T10:23:33Z","updated_at":"2018-02-09T10:23:33Z","author_association":"OWNER","body":"Hi @timbilino ,\r\n\r\nThanks for your question.\r\n\r\nI am sorry, I've never seen such an issue. It sounds like this may happen once Python uses too much memory. For an example, see this thread: https://apple.stackexchange.com/questions/167540/how-do-i-diagnose-my-python-program-being-killed-due-to-an-out-of-memory-error - other reports on \"the internet\" seems to point in the same direction.\r\n\r\nEven though the above link is in regards to MacOS, note how the questioner mentioned he/she was able to solve the problem by using a large swap in Linux.\r\n\r\nI am not sure how many files are in your data folder, but please bear in mind that TETRE has a limitation in which it loads all sentences in memory at the same time. I would suggest maybe doing a search on a subset of the data to see if the problem persists?\r\n\r\nIf it does work on a subset, I would suggest running the tool on subsets of data at once, and then merging the results elsewhere for now (might be easier doing this using the json output).\r\n\r\nHope it helps.\r\n\r\nThanks!"}}
{"actor":{"display_login":"aoldoni"}}