{"action":"created","issue":{"url":"https://api.github.com/repos/TerriaJS/terriajs/issues/1757","repository_url":"https://api.github.com/repos/TerriaJS/terriajs","labels_url":"https://api.github.com/repos/TerriaJS/terriajs/issues/1757/labels{/name}","comments_url":"https://api.github.com/repos/TerriaJS/terriajs/issues/1757/comments","events_url":"https://api.github.com/repos/TerriaJS/terriajs/issues/1757/events","html_url":"https://github.com/TerriaJS/terriajs/issues/1757","id":163993630,"number":1757,"title":"Make datasets findable on search engines","user":{"login":"meh9","id":5283003,"avatar_url":"https://avatars.githubusercontent.com/u/5283003?v=3","gravatar_id":"","url":"https://api.github.com/users/meh9","html_url":"https://github.com/meh9","followers_url":"https://api.github.com/users/meh9/followers","following_url":"https://api.github.com/users/meh9/following{/other_user}","gists_url":"https://api.github.com/users/meh9/gists{/gist_id}","starred_url":"https://api.github.com/users/meh9/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/meh9/subscriptions","organizations_url":"https://api.github.com/users/meh9/orgs","repos_url":"https://api.github.com/users/meh9/repos","events_url":"https://api.github.com/users/meh9/events{/privacy}","received_events_url":"https://api.github.com/users/meh9/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/TerriaJS/terriajs/labels/T-Enhancement","name":"T-Enhancement","color":"5555EE"}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2016-07-06T03:55:37Z","updated_at":"2016-07-07T05:50:41Z","closed_at":null,"body":"At the moment if you search for some specific dataset that you are interested in using some internet search engine, no links to that dataset on any TerriaJS instances will be found unless maybe if somebody has made a share URL and put that into a static document which was then crawled by a search engine. This is a shame because this would probably lead to a lot more users finding useful data.\r\n\r\nThe problem is that search engine crawlers can probably not \"run\" the TerriaJS instance JavaScript in order to meaningfully extract the data.\r\n\r\nWe should implement some feature that makes it possible for search engines to deep link to the layers provided on TerriaJS instances. \r\n\r\nThis might take a number of forms:\r\n\r\n* A dynamically generated page that lists all datasets with deep links (share URLs?) into the TerriaJS instance that a search engine can find and crawl.\r\n* Some form of sitemap? I know Google can accept that, but even if we made one with the share URL's of all the available layers I don't think it would be able to extract the name/metadata.\r\n* Detect when the client is a crawler and then present a \"HTML only\" version of the catalogue with all layers available together with their metadata.\r\n* Some other as-yet-unthought-of clever thing."},"comment":{"url":"https://api.github.com/repos/TerriaJS/terriajs/issues/comments/230985097","html_url":"https://github.com/TerriaJS/terriajs/issues/1757#issuecomment-230985097","issue_url":"https://api.github.com/repos/TerriaJS/terriajs/issues/1757","id":230985097,"user":{"login":"stevage","id":678344,"avatar_url":"https://avatars.githubusercontent.com/u/678344?v=3","gravatar_id":"","url":"https://api.github.com/users/stevage","html_url":"https://github.com/stevage","followers_url":"https://api.github.com/users/stevage/followers","following_url":"https://api.github.com/users/stevage/following{/other_user}","gists_url":"https://api.github.com/users/stevage/gists{/gist_id}","starred_url":"https://api.github.com/users/stevage/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/stevage/subscriptions","organizations_url":"https://api.github.com/users/stevage/orgs","repos_url":"https://api.github.com/users/stevage/repos","events_url":"https://api.github.com/users/stevage/events{/privacy}","received_events_url":"https://api.github.com/users/stevage/received_events","type":"User","site_admin":false},"created_at":"2016-07-07T05:50:41Z","updated_at":"2016-07-07T05:50:41Z","body":"Yeah, nice one. Hard, but important.\r\n\r\n* \"dynamically generated page that lists all datasets with deep links (share URLs?)\": That sounds like a pretty good outcome, and isn't harder to achieve than the others.\r\n* \"Some form of sitemap\": You'd think this would be the right answer, but yeah, without being able to provide any metadata, I can't see how it would work. It's just a list of raw URLs - where would they go to? Share URLs? GetCapabilities etc endpoints?\r\n* \"Detect when the client is a crawler and then present a HTML-only version of the catalogue\": This is essentially the same as the first option, with just a minor difference in when that HTML page is presented.\r\n* \"as-yet-unthought-of clever thing\": sure. :)\r\n\r\nBtw, I think your basic premise that Google can't run JavaScript isn't quite right: http://searchengineland.com/tested-googlebot-crawls-javascript-heres-learned-220157\r\n\r\nI suspect that even once the page is rendered, it's still not at all search engine friendly though.\r\n\r\n"}}
{"actor":{"display_login":"stevage"}}