{"action":"opened","issue":{"url":"https://api.github.com/repos/TheScienceMuseum/collectionsonline/issues/598","repository_url":"https://api.github.com/repos/TheScienceMuseum/collectionsonline","labels_url":"https://api.github.com/repos/TheScienceMuseum/collectionsonline/issues/598/labels{/name}","comments_url":"https://api.github.com/repos/TheScienceMuseum/collectionsonline/issues/598/comments","events_url":"https://api.github.com/repos/TheScienceMuseum/collectionsonline/issues/598/events","html_url":"https://github.com/TheScienceMuseum/collectionsonline/issues/598","id":191098664,"number":598,"title":"Ensure default prod. beanstalk url isn't crawled by Google.","user":{"login":"jamieu","id":91365,"avatar_url":"https://avatars.githubusercontent.com/u/91365?v=3","gravatar_id":"","url":"https://api.github.com/users/jamieu","html_url":"https://github.com/jamieu","followers_url":"https://api.github.com/users/jamieu/followers","following_url":"https://api.github.com/users/jamieu/following{/other_user}","gists_url":"https://api.github.com/users/jamieu/gists{/gist_id}","starred_url":"https://api.github.com/users/jamieu/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jamieu/subscriptions","organizations_url":"https://api.github.com/users/jamieu/orgs","repos_url":"https://api.github.com/users/jamieu/repos","events_url":"https://api.github.com/users/jamieu/events{/privacy}","received_events_url":"https://api.github.com/users/jamieu/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2016-11-22T19:17:13Z","updated_at":"2016-11-22T19:17:13Z","closed_at":null,"body":"Basically we don't want Google crawling the site via the prod.eu-west-1.elasticbeanstalk.com url.\r\n\r\nThe simplest solution would seem to be to just add a robots.txt file to exclude *.eu-west-1.elasticbeanstalk.com from crawls.\r\n\r\n> The [path] value, if specified, is to be seen relative from the root of the website for which the robots.txt file was fetched (using the same protocol, port number, host and domain names). \r\n> https://developers.google.com/webmasters/control-crawl-index/docs/robots_txt\r\n> \r\n\r\n_As an aside...I'm wondering if the way we currently have the URL/domain setup is the correct 'AWS' way of doing this (for EBS). As opposed to making the real site/domain url a CNAME for the Beanstalk (prod.eu-west-1.elasticbeanstalk.com) URL, although currently we're using a EIP mapped to the load balancer. Any thoughts, re. whether this is the correct approach or not? @nelsonic_ \r\n"}}
{"actor":{"display_login":"jamieu"}}