{"action":"created","issue":{"url":"https://api.github.com/repos/spacy-io/spaCy/issues/172","repository_url":"https://api.github.com/repos/spacy-io/spaCy","labels_url":"https://api.github.com/repos/spacy-io/spaCy/issues/172/labels{/name}","comments_url":"https://api.github.com/repos/spacy-io/spaCy/issues/172/comments","events_url":"https://api.github.com/repos/spacy-io/spaCy/issues/172/events","html_url":"https://github.com/spacy-io/spaCy/issues/172","id":115706935,"number":172,"title":"Multithreading","user":{"login":"ELind77","id":5109720,"avatar_url":"https://avatars.githubusercontent.com/u/5109720?v=3","gravatar_id":"","url":"https://api.github.com/users/ELind77","html_url":"https://github.com/ELind77","followers_url":"https://api.github.com/users/ELind77/followers","following_url":"https://api.github.com/users/ELind77/following{/other_user}","gists_url":"https://api.github.com/users/ELind77/gists{/gist_id}","starred_url":"https://api.github.com/users/ELind77/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ELind77/subscriptions","organizations_url":"https://api.github.com/users/ELind77/orgs","repos_url":"https://api.github.com/users/ELind77/repos","events_url":"https://api.github.com/users/ELind77/events{/privacy}","received_events_url":"https://api.github.com/users/ELind77/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/spacy-io/spaCy/labels/enhancement","name":"enhancement","color":"84b6eb"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":9,"created_at":"2015-11-08T03:15:24Z","updated_at":"2016-02-07T02:24:20Z","closed_at":null,"body":"I've been using Spacy for a few months now (it's amazing!) and I've added it as part of a processing pipeline I'm working on.  In order to get more throughput through the pipeline I've been using JoinableQueue and multiprocessing.Process to add parallelism.  Using this with spacy creates two difficulties:\r\n1.  Duplication of models\r\n2.  Duplication of the spacy StringStore\r\n\r\nMy understanding is that workers created with python multiprocessing have properties that are passed by value which means that the tagging/parsing models are duplicated.  You've done an incredible job making the models efficient and compact, but now I'm duplicating them all over the place and wasting your hard-won efficiency.  Is there any way that the models can be used as some kind of shared memory object?  \r\n\r\nSimilarly, it would be nice if the StringStore could be shared across workers somehow.  I'm looking into creating a generative language model based on syntactic ngrams for text classification and an efficient storage for all those ngrams would be really nice (also, if spacy can do it I don't have to work in Java :P ).\r\n\r\nI haven't worked in cython directly before but having browsed the source of this project for the last couple of months I'm not afraid to go there so long as I don't have to port my entire codebase, e.g. just make the main runner in cython and leave the rest in python.  \r\n\r\n\r\n-- Eric \r\n\r\n\r\nP.S. \r\nProbably should have started with this, but is spacy thread-safe?"},"comment":{"url":"https://api.github.com/repos/spacy-io/spaCy/issues/comments/180916769","html_url":"https://github.com/spacy-io/spaCy/issues/172#issuecomment-180916769","issue_url":"https://api.github.com/repos/spacy-io/spaCy/issues/172","id":180916769,"user":{"login":"honnibal","id":8059750,"avatar_url":"https://avatars.githubusercontent.com/u/8059750?v=3","gravatar_id":"","url":"https://api.github.com/users/honnibal","html_url":"https://github.com/honnibal","followers_url":"https://api.github.com/users/honnibal/followers","following_url":"https://api.github.com/users/honnibal/following{/other_user}","gists_url":"https://api.github.com/users/honnibal/gists{/gist_id}","starred_url":"https://api.github.com/users/honnibal/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/honnibal/subscriptions","organizations_url":"https://api.github.com/users/honnibal/orgs","repos_url":"https://api.github.com/users/honnibal/repos","events_url":"https://api.github.com/users/honnibal/events{/privacy}","received_events_url":"https://api.github.com/users/honnibal/received_events","type":"User","site_admin":false},"created_at":"2016-02-07T02:24:20Z","updated_at":"2016-02-07T02:24:20Z","body":"Some numbers on a larger job --- parsing about 10gb of text, from one month of comments from the Reddit comment corpus. The machine has 92 cores.\r\n\r\nProcesses | Threads per process | Time\r\n-------------- | ---------------------------- | -------\r\n 64             | 2                                | 99m\r\n32              | 4                                | 92m\r\n\r\nWe'll run the rest of the experiments to figure out what the best balance is, and probably make some adjustments. But it's already looking really great. If we can run at, say, 32 processes and utilize all 92 cores at full efficiency, we've effectively cut the memory use by 1/3. I'm running the 16 process, 8 threads experiment now. I have a hunch that having excess threads is good because the io, decompression, json parsing, tokenization, tagging and queueing are all done behind the GIL, so only one thread can be used while the process is setting up the work. So to keep the machine busy we have to allow some extra threads per process."}}