{"action":"created","issue":{"url":"https://api.github.com/repos/hapijs/inert/issues/60","repository_url":"https://api.github.com/repos/hapijs/inert","labels_url":"https://api.github.com/repos/hapijs/inert/issues/60/labels{/name}","comments_url":"https://api.github.com/repos/hapijs/inert/issues/60/comments","events_url":"https://api.github.com/repos/hapijs/inert/issues/60/events","html_url":"https://github.com/hapijs/inert/issues/60","id":152598042,"number":60,"title":"Hapijs directory route not being scrapped by facebook behind cloudfront","user":{"login":"ScottDowne","id":197334,"avatar_url":"https://avatars.githubusercontent.com/u/197334?v=3","gravatar_id":"","url":"https://api.github.com/users/ScottDowne","html_url":"https://github.com/ScottDowne","followers_url":"https://api.github.com/users/ScottDowne/followers","following_url":"https://api.github.com/users/ScottDowne/following{/other_user}","gists_url":"https://api.github.com/users/ScottDowne/gists{/gist_id}","starred_url":"https://api.github.com/users/ScottDowne/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ScottDowne/subscriptions","organizations_url":"https://api.github.com/users/ScottDowne/orgs","repos_url":"https://api.github.com/users/ScottDowne/repos","events_url":"https://api.github.com/users/ScottDowne/events{/privacy}","received_events_url":"https://api.github.com/users/ScottDowne/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/hapijs/inert/labels/bug","name":"bug","color":"e11d21"}],"state":"closed","locked":false,"assignee":{"login":"kanongil","id":753512,"avatar_url":"https://avatars.githubusercontent.com/u/753512?v=3","gravatar_id":"","url":"https://api.github.com/users/kanongil","html_url":"https://github.com/kanongil","followers_url":"https://api.github.com/users/kanongil/followers","following_url":"https://api.github.com/users/kanongil/following{/other_user}","gists_url":"https://api.github.com/users/kanongil/gists{/gist_id}","starred_url":"https://api.github.com/users/kanongil/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kanongil/subscriptions","organizations_url":"https://api.github.com/users/kanongil/orgs","repos_url":"https://api.github.com/users/kanongil/repos","events_url":"https://api.github.com/users/kanongil/events{/privacy}","received_events_url":"https://api.github.com/users/kanongil/received_events","type":"User","site_admin":false},"milestone":null,"comments":5,"created_at":"2016-05-02T17:13:46Z","updated_at":"2016-05-05T01:22:19Z","closed_at":"2016-05-04T09:46:44Z","body":"I initially filed this here: https://github.com/hapijs/hapi/issues/3132 Was directed here.\r\n\r\nI wonder if anyone has hit this?\r\n\r\nI have this https://donate.mozilla.org/en-US/\r\n\r\nWhich is a hapi server. In this case it's serving a static html file: https://github.com/mozilla/donate.mozilla.org/blob/master/server.js#L352-L358\r\n\r\n```\r\n server.route([{\r\n      method: 'GET',\r\n      path: '/{params*}',\r\n      handler: {\r\n        directory: {\r\n          path: Path.join(__dirname, 'public')\r\n        }\r\n      }\r\n}\r\n```\r\n\r\nSeems to work fine as a file server. However, when it interacts with cloudfront  and facebooks scrapper, something breaks. Not fully understanding what's happening, but what I can piece together is:\r\n\r\nThe hapi server sends the file contents as `Transfer-Encoding: chunked`\r\n\r\nCloudfront then has \"If the viewer makes a Range GET request and the origin returns Transfer-Encoding: chunked, CloudFront returns the entire object to the viewer instead of the requested range.\" from http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/RangeGETs.html\r\n\r\nFacebook's scrapper then chokes on the size of the range not being expected.\r\n\r\nYou can test that here: https://developers.facebook.com/tools/debug/og/object/\r\n\r\nPaste in: http://donate.mozilla.org/en-US/thunderbird/\r\n\r\nThen click \"fetch new scrape information\"\r\n\r\nFacebook has provided me with a curl command that simulates what their scrapper does:\r\n\r\n```\r\ncurl -G -v --compressed -H \"Range: bytes=0-500000\" -H \"Connection: close\" -A \"facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)\" \"https://donate.mozilla.org/en-US/thunderbird/share/\"\r\n```\r\n\r\nIt responds with `curl: (18) transfer closed with 4317 bytes remaining to read`\r\n\r\nIt also doesn't respond with `Transfer-Encoding: chunked`\r\n\r\nIf I curl directly to the server without cloudfront:\r\n\r\n```\r\ncurl -G -vv --compressed -H \"Range: bytes=0-500000\" -H \"Connection: close\" -A \"facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)\" \"https://donate-mozilla-org-us-prod.herokuapp.com/en-US/\"\r\n```\r\n\r\nI get back `Transfer-Encoding: chunked` which I then think cloudfront returns another thing to facebook which facebook doesn't expect.\r\n\r\nThoughts? Can I just turn off `Transfer-Encoding: chunked` and how would I do that with the static directory server?"},"comment":{"url":"https://api.github.com/repos/hapijs/inert/issues/comments/217054046","html_url":"https://github.com/hapijs/inert/issues/60#issuecomment-217054046","issue_url":"https://api.github.com/repos/hapijs/inert/issues/60","id":217054046,"user":{"login":"ScottDowne","id":197334,"avatar_url":"https://avatars.githubusercontent.com/u/197334?v=3","gravatar_id":"","url":"https://api.github.com/users/ScottDowne","html_url":"https://github.com/ScottDowne","followers_url":"https://api.github.com/users/ScottDowne/followers","following_url":"https://api.github.com/users/ScottDowne/following{/other_user}","gists_url":"https://api.github.com/users/ScottDowne/gists{/gist_id}","starred_url":"https://api.github.com/users/ScottDowne/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ScottDowne/subscriptions","organizations_url":"https://api.github.com/users/ScottDowne/orgs","repos_url":"https://api.github.com/users/ScottDowne/repos","events_url":"https://api.github.com/users/ScottDowne/events{/privacy}","received_events_url":"https://api.github.com/users/ScottDowne/received_events","type":"User","site_admin":false},"created_at":"2016-05-05T01:22:19Z","updated_at":"2016-05-05T01:22:19Z","body":"Fantastic! :tada: I pulled in the changes today, worked like a charm! Thanks very much for the quick response! \r\n\r\nIt's not shipped to our prod site yet, but I'll do that tomorrow. "}}