{"action":"opened","issue":{"url":"https://api.github.com/repos/scrapy/scrapy/issues/1770","repository_url":"https://api.github.com/repos/scrapy/scrapy","labels_url":"https://api.github.com/repos/scrapy/scrapy/issues/1770/labels{/name}","comments_url":"https://api.github.com/repos/scrapy/scrapy/issues/1770/comments","events_url":"https://api.github.com/repos/scrapy/scrapy/issues/1770/events","html_url":"https://github.com/scrapy/scrapy/issues/1770","id":131928704,"number":1770,"title":"PY3: Fail to download the second or later requests to hosts using secure cookies","user":{"login":"orangain","id":532251,"avatar_url":"https://avatars.githubusercontent.com/u/532251?v=3","gravatar_id":"","url":"https://api.github.com/users/orangain","html_url":"https://github.com/orangain","followers_url":"https://api.github.com/users/orangain/followers","following_url":"https://api.github.com/users/orangain/following{/other_user}","gists_url":"https://api.github.com/users/orangain/gists{/gist_id}","starred_url":"https://api.github.com/users/orangain/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/orangain/subscriptions","organizations_url":"https://api.github.com/users/orangain/orgs","repos_url":"https://api.github.com/users/orangain/repos","events_url":"https://api.github.com/users/orangain/events{/privacy}","received_events_url":"https://api.github.com/users/orangain/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":0,"created_at":"2016-02-07T03:18:58Z","updated_at":"2016-02-07T03:18:58Z","closed_at":null,"body":"## Environment\r\n\r\n* Mac OS X 10.10.5\r\n* Python 3.4.2\r\n* Scrapy 1.1.0rc1\r\n* Twisted 15.5.0\r\n\r\n## Steps to Reproduce\r\n\r\n1. Save the following spider as `secure_cookie_spider.py`.\r\n\r\n   ```py\r\n   import scrapy\r\n   \r\n   \r\n   class SecureCookieSpider(scrapy.Spider):\r\n       name = 'secure_cookie_spider'\r\n       start_urls = [\r\n           'https://github.com/',\r\n       ]\r\n   \r\n       def parse(self, response):\r\n           # Request the same url again\r\n           yield scrapy.Request(url=response.url, callback=self.parse_second_request)\r\n   \r\n       def parse_second_request(self, response):\r\n           pass\r\n   ```\r\n\r\n2. Run the following command.\r\n\r\n   ```\r\n   $ scrapy runspider secure_cookie_spider.py\r\n   ```\r\n\r\n## Expected Results\r\n\r\nNo error is reported.\r\n\r\n## Actual Results\r\n\r\nFail to download the second request with `AttributeError: 'WrappedRequest' object has no attribute 'type'`.\r\n\r\n```\r\n$ scrapy runspider secure_cookie_spider.py\r\n2016-02-07 11:57:11 [scrapy] INFO: Scrapy 1.1.0rc1 started (bot: scrapybot)\r\n2016-02-07 11:57:11 [scrapy] INFO: Overridden settings: {}\r\n2016-02-07 11:57:11 [scrapy] INFO: Enabled extensions:\r\n['scrapy.extensions.corestats.CoreStats',\r\n 'scrapy.extensions.logstats.LogStats']\r\n2016-02-07 11:57:11 [scrapy] INFO: Enabled downloader middlewares:\r\n['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',\r\n 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',\r\n 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',\r\n 'scrapy.downloadermiddlewares.retry.RetryMiddleware',\r\n 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',\r\n 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',\r\n 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',\r\n 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',\r\n 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',\r\n 'scrapy.downloadermiddlewares.chunked.ChunkedTransferMiddleware',\r\n 'scrapy.downloadermiddlewares.stats.DownloaderStats']\r\n2016-02-07 11:57:11 [scrapy] INFO: Enabled spider middlewares:\r\n['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',\r\n 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',\r\n 'scrapy.spidermiddlewares.referer.RefererMiddleware',\r\n 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',\r\n 'scrapy.spidermiddlewares.depth.DepthMiddleware']\r\n2016-02-07 11:57:11 [scrapy] INFO: Enabled item pipelines:\r\n[]\r\n2016-02-07 11:57:11 [scrapy] INFO: Spider opened\r\n2016-02-07 11:57:11 [scrapy] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)\r\n2016-02-07 11:57:12 [scrapy] DEBUG: Crawled (200) <GET https://github.com/> (referer: None)\r\n2016-02-07 11:57:12 [scrapy] ERROR: Error downloading <GET https://github.com/>\r\nTraceback (most recent call last):\r\n  File \"/private/tmp/scrapy1.1/venv/lib/python3.4/site-packages/twisted/internet/defer.py\", line 1128, in _inlineCallbacks\r\n    result = g.send(result)\r\n  File \"/private/tmp/scrapy1.1/venv/lib/python3.4/site-packages/scrapy/core/downloader/middleware.py\", line 37, in process_request\r\n    response = yield method(request=request, spider=spider)\r\n  File \"/private/tmp/scrapy1.1/venv/lib/python3.4/site-packages/scrapy/downloadermiddlewares/cookies.py\", line 39, in process_request\r\n    jar.add_cookie_header(request)\r\n  File \"/private/tmp/scrapy1.1/venv/lib/python3.4/site-packages/scrapy/http/cookies.py\", line 42, in add_cookie_header\r\n    cookies += self.jar._cookies_for_domain(host, wreq)\r\n  File \"/usr/local/Cellar/python3/3.4.2_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/cookiejar.py\", line 1242, in _cookies_for_domain\r\n    if not self._policy.return_ok(cookie, request):\r\n  File \"/usr/local/Cellar/python3/3.4.2_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/cookiejar.py\", line 1077, in return_ok\r\n    if not fn(cookie, request):\r\n  File \"/usr/local/Cellar/python3/3.4.2_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/cookiejar.py\", line 1103, in return_ok_secure\r\n    if cookie.secure and request.type != \"https\":\r\nAttributeError: 'WrappedRequest' object has no attribute 'type'\r\n2016-02-07 11:57:12 [scrapy] INFO: Closing spider (finished)\r\n2016-02-07 11:57:12 [scrapy] INFO: Dumping Scrapy stats:\r\n{'downloader/exception_count': 1,\r\n 'downloader/exception_type_count/builtins.AttributeError': 1,\r\n 'downloader/request_bytes': 211,\r\n 'downloader/request_count': 1,\r\n 'downloader/request_method_count/GET': 1,\r\n 'downloader/response_bytes': 9735,\r\n 'downloader/response_count': 1,\r\n 'downloader/response_status_count/200': 1,\r\n 'finish_reason': 'finished',\r\n 'finish_time': datetime.datetime(2016, 2, 7, 2, 57, 12, 757829),\r\n 'log_count/DEBUG': 1,\r\n 'log_count/ERROR': 1,\r\n 'log_count/INFO': 7,\r\n 'request_depth_max': 1,\r\n 'response_received_count': 1,\r\n 'scheduler/dequeued': 2,\r\n 'scheduler/dequeued/memory': 2,\r\n 'scheduler/enqueued': 2,\r\n 'scheduler/enqueued/memory': 2,\r\n 'start_time': datetime.datetime(2016, 2, 7, 2, 57, 11, 384330)}\r\n2016-02-07 11:57:12 [scrapy] INFO: Spider closed (finished)\r\n```\r\n\r\nNote that no error is reported in Python 2."}}