{"action":"created","issue":{"url":"https://api.github.com/repos/eggpi/citationhunt/issues/9","repository_url":"https://api.github.com/repos/eggpi/citationhunt","labels_url":"https://api.github.com/repos/eggpi/citationhunt/issues/9/labels{/name}","comments_url":"https://api.github.com/repos/eggpi/citationhunt/issues/9/comments","events_url":"https://api.github.com/repos/eggpi/citationhunt/issues/9/events","html_url":"https://github.com/eggpi/citationhunt/issues/9","id":59914541,"number":9,"title":"Alternative approach using live database","user":{"login":"zhaofengli","id":2189609,"avatar_url":"https://avatars.githubusercontent.com/u/2189609?v=3","gravatar_id":"","url":"https://api.github.com/users/zhaofengli","html_url":"https://github.com/zhaofengli","followers_url":"https://api.github.com/users/zhaofengli/followers","following_url":"https://api.github.com/users/zhaofengli/following{/other_user}","gists_url":"https://api.github.com/users/zhaofengli/gists{/gist_id}","starred_url":"https://api.github.com/users/zhaofengli/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zhaofengli/subscriptions","organizations_url":"https://api.github.com/users/zhaofengli/orgs","repos_url":"https://api.github.com/users/zhaofengli/repos","events_url":"https://api.github.com/users/zhaofengli/events{/privacy}","received_events_url":"https://api.github.com/users/zhaofengli/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":4,"created_at":"2015-03-05T07:16:15Z","updated_at":"2016-03-30T12:38:48Z","closed_at":"2016-03-30T12:38:48Z","body":"In case you are not aware, WMF provides [Tool Labs](https://wikitech.wikimedia.org/wiki/Help:Tool Labs) which is a free hosting service for Wikimedia-related tools, with access to live database replicas and latest dumps. With live database access, I propose another way to discover pages needing citations:\r\n1. Using SQL, select a random page in article space (namespace 0) which transcludes {{cn}} with the `templatelinks` table (see [database layout](https://www.mediawiki.org/wiki/Manual:Database_layout) and [example](http://quarry.wmflabs.org/query/2420))\r\n2. Obtain the page content via API (There doesn't seem to be a way to get this from the database)\r\n3. Parse the page using existing code, to get the uncited part\r\n\r\nThis way, we can avoid the lag caused by the use of dumps (what is shown represents the current state of the article) and there will be no need to pre-process the dumps. Editors also tend to trust tools hosted on Tool Labs more than those on third-party servers.\r\n\r\nTools on Tool Labs need to be licensed under a [OSI-approved license](http://opensource.org/licenses/).\r\n\r\nZhaofeng Li ([User:Zhaofeng Li](https://en.wikipedia.org/wiki/User:Zhaofeng_Li) on Wikipedia)"},"comment":{"url":"https://api.github.com/repos/eggpi/citationhunt/issues/comments/203412265","html_url":"https://github.com/eggpi/citationhunt/issues/9#issuecomment-203412265","issue_url":"https://api.github.com/repos/eggpi/citationhunt/issues/9","id":203412265,"user":{"login":"eggpi","id":489134,"avatar_url":"https://avatars.githubusercontent.com/u/489134?v=3","gravatar_id":"","url":"https://api.github.com/users/eggpi","html_url":"https://github.com/eggpi","followers_url":"https://api.github.com/users/eggpi/followers","following_url":"https://api.github.com/users/eggpi/following{/other_user}","gists_url":"https://api.github.com/users/eggpi/gists{/gist_id}","starred_url":"https://api.github.com/users/eggpi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/eggpi/subscriptions","organizations_url":"https://api.github.com/users/eggpi/orgs","repos_url":"https://api.github.com/users/eggpi/repos","events_url":"https://api.github.com/users/eggpi/events{/privacy}","received_events_url":"https://api.github.com/users/eggpi/received_events","type":"User","site_admin":false},"created_at":"2016-03-30T12:38:48Z","updated_at":"2016-03-30T12:38:48Z","body":"After 7a0e9dfd40352595f0b6d83e243c9c140036611f, we're (finally!) using the live databases and API and no longer depend on dumps :)"}}