{"action":"created","issue":{"url":"https://api.github.com/repos/explosion/spaCy/issues/1962","repository_url":"https://api.github.com/repos/explosion/spaCy","labels_url":"https://api.github.com/repos/explosion/spaCy/issues/1962/labels{/name}","comments_url":"https://api.github.com/repos/explosion/spaCy/issues/1962/comments","events_url":"https://api.github.com/repos/explosion/spaCy/issues/1962/events","html_url":"https://github.com/explosion/spaCy/issues/1962","id":295937596,"number":1962,"title":"en tokenizer model not loading if spacy installed in user home","user":{"login":"nkrot","id":1128218,"avatar_url":"https://avatars3.githubusercontent.com/u/1128218?v=4","gravatar_id":"","url":"https://api.github.com/users/nkrot","html_url":"https://github.com/nkrot","followers_url":"https://api.github.com/users/nkrot/followers","following_url":"https://api.github.com/users/nkrot/following{/other_user}","gists_url":"https://api.github.com/users/nkrot/gists{/gist_id}","starred_url":"https://api.github.com/users/nkrot/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nkrot/subscriptions","organizations_url":"https://api.github.com/users/nkrot/orgs","repos_url":"https://api.github.com/users/nkrot/repos","events_url":"https://api.github.com/users/nkrot/events{/privacy}","received_events_url":"https://api.github.com/users/nkrot/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-02-09T16:52:51Z","updated_at":"2018-02-09T17:10:35Z","closed_at":null,"author_association":"NONE","body":"I want to try spacy tokenizer for English. So, I have installed spacy in user space with\r\n\r\n    > pip3 install spacy --user\r\n\r\nthat leaves me with\r\n\r\n    > python3 -m spacy validate\r\n\r\n    Installed models (spaCy v2.0.7)\r\n    $HOME/.local/lib/python3.6/site-packages/spacy\r\n\r\nMy guess is that the language-specific models are here: `$HOME/.local/lib/python3.6/site-packages/spacy/lang/en`. This directory is not empty, so i guess `en` models have been installed, at least i see there a file named `tokenizer_exceptions.py`.\r\n\r\nHowever, when using spacy in a script, i receive an error:\r\n```\r\n>>> import spacy\r\n>>> nlp = spacy.load(\"en\")\r\nTraceback (most recent call last):\r\n  File \"<stdin>\", line 1, in <module>\r\n  File \"$HOME/.local/lib/python3.6/site-packages/spacy/__init__.py\", line 19, in load\r\n    return util.load_model(name, **overrides)\r\n  File \"$HOME/.local/lib/python3.6/site-packages/spacy/util.py\", line 119, in load_model\r\n    raise IOError(\"Can't find model '%s'\" % name)\r\nOSError: Can't find model 'en'\r\n```\r\n\r\nIf I try to install language models like follows:\r\n\r\n    python3 -m spacy download en\r\n\r\nthe installer tries to put data into system-wide location, which is not desired\r\n\r\n    error: could not create '/usr/local/lib/python3.6/site-packages/en_core_web_sm': Permission denied\r\n\r\nBasically, two questions: Were the language models installed? And if not, how to install them in the user home under `.local/` where `pip` installs stuff?\r\n\r\n## Info about spaCy\r\n\r\n* **spaCy version:** 2.0.7\r\n* **Platform:** Linux-3.10.0-327.22.2.el7.x86_64-x86_64-with-centos-7.2.1511-Core\r\n* **Python version:** 3.6.4\r\n\r\nThanks in advance,\r\n"},"comment":{"url":"https://api.github.com/repos/explosion/spaCy/issues/comments/364496271","html_url":"https://github.com/explosion/spaCy/issues/1962#issuecomment-364496271","issue_url":"https://api.github.com/repos/explosion/spaCy/issues/1962","id":364496271,"user":{"login":"ines","id":13643239,"avatar_url":"https://avatars0.githubusercontent.com/u/13643239?v=4","gravatar_id":"","url":"https://api.github.com/users/ines","html_url":"https://github.com/ines","followers_url":"https://api.github.com/users/ines/followers","following_url":"https://api.github.com/users/ines/following{/other_user}","gists_url":"https://api.github.com/users/ines/gists{/gist_id}","starred_url":"https://api.github.com/users/ines/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ines/subscriptions","organizations_url":"https://api.github.com/users/ines/orgs","repos_url":"https://api.github.com/users/ines/repos","events_url":"https://api.github.com/users/ines/events{/privacy}","received_events_url":"https://api.github.com/users/ines/received_events","type":"User","site_admin":false},"created_at":"2018-02-09T17:10:35Z","updated_at":"2018-02-09T17:10:35Z","author_association":"OWNER","body":"spaCy models are Python packages and when you run the `spacy download` command, the respective URL to the pip-installable file is fetched from our [models repo](https://github.com/explosion/spacy-models) and downloaded via pip in a subprocess. I think you might be experiencing the same issue as #1220 (also see #1456), since the `--user` flag isn't set in the subprocess run by spaCy.\r\n\r\nThe easiest workaround for now would be to [install the model manually](https://spacy.io/usage/models#download-pip). For example, [here's the download of the small English model `en_core_web_sm`](https://github.com/explosion/spacy-models/releases/tag/en_core_web_sm-2.0.0) (which is the default model that's also downloaded via `spacy download en`). The attached `.tar.gz` archive is an installable package, so you should be able to run:\r\n\r\n```bash\r\npip install /path/to/en_core_web_sm-2.0.0.tar.gz --user\r\n```\r\n\r\nThis will let you load the model via `spacy.load('en_core_web_sm')`. [Here's some more info](https://spacy.io/usage/models#usage) on how model loading works.\r\n\r\nThe content in `spacy.lang.en` is the English *language data*, i.e. the tokenizer exceptions, punctuation rules, lemmatizer data etc. All spaCy models specify a language code (e.g. `\"en\"`) that tells spaCy which language data to initialise. So when you load a model, spaCy will create the language class and processing pipeline, and load in the binary weights so you can use the features that require a statistical model – like the tagger, parser or entity recognizer. [Here's](https://spacy.io/usage/processing-pipelines#pipelines) a more detailed explanation of this.\r\n\r\nIf you only want to use the tokenizer, you don't even have to download and load a model – instead, you can simply import and use the respective language class:\r\n\r\n```python\r\nfrom spacy.lang.en import English\r\n\r\nnlp = English()\r\ndoc = nlp(u\"This text will be tokenized but won't have POS tags, dependencies or entities\")\r\n``` "}}
{"actor":{"display_login":"ines"}}