{"action":"created","issue":{"url":"https://api.github.com/repos/scrapy/scrapy/issues/1908","repository_url":"https://api.github.com/repos/scrapy/scrapy","labels_url":"https://api.github.com/repos/scrapy/scrapy/issues/1908/labels{/name}","comments_url":"https://api.github.com/repos/scrapy/scrapy/issues/1908/comments","events_url":"https://api.github.com/repos/scrapy/scrapy/issues/1908/events","html_url":"https://github.com/scrapy/scrapy/issues/1908","id":146575064,"number":1908,"title":"How to work with a very large “allowed_domains” attribute in scrapy?","user":{"login":"15310944349","id":11782806,"avatar_url":"https://avatars.githubusercontent.com/u/11782806?v=3","gravatar_id":"","url":"https://api.github.com/users/15310944349","html_url":"https://github.com/15310944349","followers_url":"https://api.github.com/users/15310944349/followers","following_url":"https://api.github.com/users/15310944349/following{/other_user}","gists_url":"https://api.github.com/users/15310944349/gists{/gist_id}","starred_url":"https://api.github.com/users/15310944349/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/15310944349/subscriptions","organizations_url":"https://api.github.com/users/15310944349/orgs","repos_url":"https://api.github.com/users/15310944349/repos","events_url":"https://api.github.com/users/15310944349/events{/privacy}","received_events_url":"https://api.github.com/users/15310944349/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":5,"created_at":"2016-04-07T10:32:29Z","updated_at":"2016-04-11T03:13:51Z","closed_at":null,"body":"Because the allowed_domains is very big, it throws this exception:\r\n`regex = r'^(.*.)?(%s)$' % '|'.join(re.escape(d) for d in allowed_domains if d is not None)\r\n`\r\nHow do I solve this problem?"},"comment":{"url":"https://api.github.com/repos/scrapy/scrapy/issues/comments/208143054","html_url":"https://github.com/scrapy/scrapy/issues/1908#issuecomment-208143054","issue_url":"https://api.github.com/repos/scrapy/scrapy/issues/1908","id":208143054,"user":{"login":"15310944349","id":11782806,"avatar_url":"https://avatars.githubusercontent.com/u/11782806?v=3","gravatar_id":"","url":"https://api.github.com/users/15310944349","html_url":"https://github.com/15310944349","followers_url":"https://api.github.com/users/15310944349/followers","following_url":"https://api.github.com/users/15310944349/following{/other_user}","gists_url":"https://api.github.com/users/15310944349/gists{/gist_id}","starred_url":"https://api.github.com/users/15310944349/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/15310944349/subscriptions","organizations_url":"https://api.github.com/users/15310944349/orgs","repos_url":"https://api.github.com/users/15310944349/repos","events_url":"https://api.github.com/users/15310944349/events{/privacy}","received_events_url":"https://api.github.com/users/15310944349/received_events","type":"User","site_admin":false},"created_at":"2016-04-11T03:13:51Z","updated_at":"2016-04-11T03:13:51Z","body":"@redapple `https://stackoverflow.com/questions/36440681/how-to-work-with-a-very-large-allowed-domains-attribute-in-scrapy/36454033` the question is my question, but the method efficienty is too low"}}