{"action":"created","issue":{"url":"https://api.github.com/repos/RasaHQ/rasa_nlu/issues/1204","repository_url":"https://api.github.com/repos/RasaHQ/rasa_nlu","labels_url":"https://api.github.com/repos/RasaHQ/rasa_nlu/issues/1204/labels{/name}","comments_url":"https://api.github.com/repos/RasaHQ/rasa_nlu/issues/1204/comments","events_url":"https://api.github.com/repos/RasaHQ/rasa_nlu/issues/1204/events","html_url":"https://github.com/RasaHQ/rasa_nlu/issues/1204","id":338005425,"node_id":"MDU6SXNzdWUzMzgwMDU0MjU=","number":1204,"title":"Running out of memory very quickly during training","user":{"login":"ruze00","id":3300000,"node_id":"MDQ6VXNlcjMzMDAwMDA=","avatar_url":"https://avatars1.githubusercontent.com/u/3300000?v=4","gravatar_id":"","url":"https://api.github.com/users/ruze00","html_url":"https://github.com/ruze00","followers_url":"https://api.github.com/users/ruze00/followers","following_url":"https://api.github.com/users/ruze00/following{/other_user}","gists_url":"https://api.github.com/users/ruze00/gists{/gist_id}","starred_url":"https://api.github.com/users/ruze00/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ruze00/subscriptions","organizations_url":"https://api.github.com/users/ruze00/orgs","repos_url":"https://api.github.com/users/ruze00/repos","events_url":"https://api.github.com/users/ruze00/events{/privacy}","received_events_url":"https://api.github.com/users/ruze00/received_events","type":"User","site_admin":false},"labels":[{"id":461549567,"node_id":"MDU6TGFiZWw0NjE1NDk1Njc=","url":"https://api.github.com/repos/RasaHQ/rasa_nlu/labels/question","name":"question","color":"cc317c","default":true}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2018-07-03T18:16:02Z","updated_at":"2018-07-10T16:02:40Z","closed_at":null,"author_association":"NONE","body":"The job I'm trying to run has a lot of data and it runs out of physical memory very quickly using the `tensorflow_embedding` pipeline.  Is there any way to configure the components not to use memory exclusively?\r\n\r\nAlternatively, could another solution be to configure or modify the pipeline to support distributed Tensorflow?  Any tips on how to make this happen?\r\n\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/RasaHQ/rasa_nlu/issues/comments/403876796","html_url":"https://github.com/RasaHQ/rasa_nlu/issues/1204#issuecomment-403876796","issue_url":"https://api.github.com/repos/RasaHQ/rasa_nlu/issues/1204","id":403876796,"node_id":"MDEyOklzc3VlQ29tbWVudDQwMzg3Njc5Ng==","user":{"login":"ruze00","id":3300000,"node_id":"MDQ6VXNlcjMzMDAwMDA=","avatar_url":"https://avatars1.githubusercontent.com/u/3300000?v=4","gravatar_id":"","url":"https://api.github.com/users/ruze00","html_url":"https://github.com/ruze00","followers_url":"https://api.github.com/users/ruze00/followers","following_url":"https://api.github.com/users/ruze00/following{/other_user}","gists_url":"https://api.github.com/users/ruze00/gists{/gist_id}","starred_url":"https://api.github.com/users/ruze00/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ruze00/subscriptions","organizations_url":"https://api.github.com/users/ruze00/orgs","repos_url":"https://api.github.com/users/ruze00/repos","events_url":"https://api.github.com/users/ruze00/events{/privacy}","received_events_url":"https://api.github.com/users/ruze00/received_events","type":"User","site_admin":false},"created_at":"2018-07-10T16:02:40Z","updated_at":"2018-07-10T16:02:40Z","author_association":"NONE","body":"I'm using a few hundred thousand utterances with a little more than a thousand intents.  The amount of RAM is 196 GB on that server.\r\n\r\nHowever, for our purposes I have worked around some of the memory issues by removing the call to `deepcopy` of the working data in `model.py`, which is pointless for the operations we're using, and other optimizations including removing the `toarray()` conversion from a sparse array in the count vectorizer."}}
{"actor":{"display_login":"ruze00"}}