{"action":"created","comment":{"url":"https://api.github.com/repos/mozilla-services/amo2kinto/pulls/comments/62817218","id":62817218,"diff_hunk":"@@ -1,23 +1,150 @@\n-kinto2xml\n----------\n+#########\n+amo2kinto\n+#########\n \n-Generate a blocklist.xml file from Kinto collections.\n+\n+Installation\n+============\n+\n+To install the release, just use::\n+\n+    pip install amo2kinto\n+\n+\n+To install the development environment, just run::\n+\n+    make install\n+\n+\n+System dependencies\n+-------------------\n+\n+In order to build ``amo2kinto`` dependencies you may need the following libraries:\n+\n+- Building **lxml** will require: ``libxml2-dev libxslt-dev python-dev``\n+- Building **cryptography** will require: ``build-essential libffi-dev libssl-dev python-dev``\n+\n+If you have got error messages telling you a header file is missing, please install them:\n+\n+- ``libxml/xmlversion.h: File not found``\n+- ``ffi.h: File not found``\n+- ``ssl.h: File not found``\n+\n+\n+Importing blocklists from AMO into Kinto\n+========================================\n+\n+The `addons-server <https://github.com/mozilla/addons-server/>`_ is\n+able to export its blocklists database in JSON.\n+\n+You can use the ``json2kinto`` script to load this database into a\n+kinto server::\n+\n+    json2kinto --server http://localhost:8888/v1 \\\n+               --amo-server https://addons.mozilla.org/  \\\n+               --schema-file schemas.json\n+\n+The last version of the schema file can be find here:\n+https://github.com/mozilla-services/amo-blocklist-ui/blob/master/amo-blocklist.json\n+\n+The script is able to synchronize (add new blocked items and remove old ones).\n+\n+However if a blocked item already exists it won't be altered.\n+\n+**json2kinto** gives you the ability to configure what you want to\n+load, with which user credentials and in which bucket and collection.\n \n .. code-block::\n \n-    usage: kinto2xml [-h] [-s SERVER] [-a AUTH] [-b BUCKET] [-v] [-q] [-D]\n-                     [-o OUT]\n-\n-    Build a blocklists.xml file.\n-\n-    optional arguments:\n-      -h, --help            show this help message and exit\n-      -s SERVER, --server SERVER\n-                            The location of the remote server (with prefix)\n-      -a AUTH, --auth AUTH  BasicAuth token:my-secret\n-      -b BUCKET, --bucket BUCKET\n-                            Bucket name.\n-      -v, --verbose         Show all messages.\n-      -q, --quiet           Show only critical errors.\n-      -D, --debug           Show all messages, including debug messages.\n-      -o OUT, --out OUT     Output file.\n+  -h, --help            show this help message and exit\n+  --amo-server AMO_SERVER\n+                        The AMO server to import from\n+  -s SERVER, --server SERVER\n+                        The location of the remote server (with prefix)\n+  -a AUTH, --auth AUTH  BasicAuth token:my-secret\n+  -v, --verbose         Show all messages.\n+  -q, --quiet           Show only critical errors.\n+  -D, --debug           Show all messages, including debug messages.\n+  -S SCHEMA_FILE, --schema-file SCHEMA_FILE\n+                        JSON Schemas file\n+  --no-schema           Should we handle schemas\n+  --certificates-bucket CERTIFICATES_BUCKET\n+                        Bucket name for certificates\n+  --certificates-collection CERTIFICATES_COLLECTION\n+                        Collection name for certificates\n+  --gfx-bucket GFX_BUCKET\n+                        Bucket name for gfx\n+  --gfx-collection GFX_COLLECTION\n+                        Collection name for gfx\n+  --addons-bucket ADDONS_BUCKET\n+                        Bucket name for addons\n+  --addons-collection ADDONS_COLLECTION\n+                        Collection name for addon\n+  --plugins-bucket PLUGINS_BUCKET\n+                        Bucket name for plugins\n+  --plugins-collection PLUGINS_COLLECTION\n+                        Collection name for plugin\n+  -C, --certificates    Only import certificates\n+  -G, --gfx             Only import GFX drivers\n+  -A, --addons          Only import addons\n+  -P, --plugins         Only import plugins\n+\n+\n+\n+Generate a blocklist.xml file from Kinto collections\n+====================================================\n+\n+If you want to export blocklists stored in Kinto in the AMO XML export\n+format, you can use the ``kinto2xml`` script::\n+\n+    kinto2xml -s http://localhost:8888\n+\n+\n+**kinto2xml** gives you the ability to configure what you want to\n+export and in which bucket and collection are the data stored.\n+\n+.. code-block::\n+\n+  -h, --help            show this help message and exit\n+  -s SERVER, --server SERVER\n+                        The location of the remote server (with prefix)\n+  -a AUTH, --auth AUTH  BasicAuth token:my-secret\n+  -v, --verbose         Show all messages.\n+  -q, --quiet           Show only critical errors.\n+  -D, --debug           Show all messages, including debug messages.\n+  --certificates-bucket CERTIFICATES_BUCKET\n+                        Bucket name for certificates\n+  --certificates-collection CERTIFICATES_COLLECTION\n+                        Collection name for certificates\n+  --gfx-bucket GFX_BUCKET\n+                        Bucket name for gfx\n+  --gfx-collection GFX_COLLECTION\n+                        Collection name for gfx\n+  --addons-bucket ADDONS_BUCKET\n+                        Bucket name for addons\n+  --addons-collection ADDONS_COLLECTION\n+                        Collection name for addon\n+  --plugins-bucket PLUGINS_BUCKET\n+                        Bucket name for plugins\n+  --plugins-collection PLUGINS_COLLECTION\n+                        Collection name for plugin\n+  --app APP             Targeted blocklists.xml APP id\n+  -o OUT, --out OUT     Output XML file.\n+\n+\n+Show differences between two XML files\n+======================================\n+\n+Once you've imported the data and exported them, you may want to\n+validate that both Kinto and AMO export the exact same data in XML.\n+\n+You can use the ``xml-verifier`` command to validate that both files\n+are rendered the same way::\n+\n+    xml-verifier \\\n+        https://blocklist.addons.mozilla.org/blocklist/3/{ec8030f7-c20a-464f-9b0e-13a3a9e97384}/46.0/ \\\n+        https://kinto-reader.dev.mozaws.net/v1/blocklist/3/{ec8030f7-c20a-464f-9b0e-13a3a9e97384}/46.0/","path":"README.rst","position":196,"original_position":165,"commit_id":"7c0d8dad7955c7878188b096b69f13aa626b0ef4","original_commit_id":"a9d6899a2dd4e00b31eb134cb096905056eae876","user":{"login":"leplatrem","id":546692,"avatar_url":"https://avatars.githubusercontent.com/u/546692?v=3","gravatar_id":"","url":"https://api.github.com/users/leplatrem","html_url":"https://github.com/leplatrem","followers_url":"https://api.github.com/users/leplatrem/followers","following_url":"https://api.github.com/users/leplatrem/following{/other_user}","gists_url":"https://api.github.com/users/leplatrem/gists{/gist_id}","starred_url":"https://api.github.com/users/leplatrem/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/leplatrem/subscriptions","organizations_url":"https://api.github.com/users/leplatrem/orgs","repos_url":"https://api.github.com/users/leplatrem/repos","events_url":"https://api.github.com/users/leplatrem/events{/privacy}","received_events_url":"https://api.github.com/users/leplatrem/received_events","type":"User","site_admin":false},"body":"What is this URL ? \r\nI suggest that you use the local files first, and then mention that it supports a remote URL of a Kinto server with the kinto-amo plugin install.\r\n\r\nWould be nice to give the equivalent kinto2xml command along: \r\n\r\n```\r\nAPPID=\"{ec8030f7-c20a-464f-9b0e-13a3a9e97384}\"\r\nkinto2xml --app $APPID -o generated-blocklists.xml\r\nxml-verifier https://blocklist.addons.mozilla.org/blocklist/3/$APPID/46.0/ generated-blocklists.xml\r\n```","created_at":"2016-05-11T09:29:10Z","updated_at":"2016-05-11T09:29:10Z","html_url":"https://github.com/mozilla-services/amo2kinto/pull/23#discussion_r62817218","pull_request_url":"https://api.github.com/repos/mozilla-services/amo2kinto/pulls/23","_links":{"self":{"href":"https://api.github.com/repos/mozilla-services/amo2kinto/pulls/comments/62817218"},"html":{"href":"https://github.com/mozilla-services/amo2kinto/pull/23#discussion_r62817218"},"pull_request":{"href":"https://api.github.com/repos/mozilla-services/amo2kinto/pulls/23"}}},"pull_request":{"url":"https://api.github.com/repos/mozilla-services/amo2kinto/pulls/23","id":69638783,"html_url":"https://github.com/mozilla-services/amo2kinto/pull/23","diff_url":"https://github.com/mozilla-services/amo2kinto/pull/23.diff","patch_url":"https://github.com/mozilla-services/amo2kinto/pull/23.patch","issue_url":"https://api.github.com/repos/mozilla-services/amo2kinto/issues/23","number":23,"state":"open","locked":false,"title":"Add documentation about json2kinto, kinto2xml and xml-verifier commands.","user":{"login":"Natim","id":229453,"avatar_url":"https://avatars.githubusercontent.com/u/229453?v=3","gravatar_id":"","url":"https://api.github.com/users/Natim","html_url":"https://github.com/Natim","followers_url":"https://api.github.com/users/Natim/followers","following_url":"https://api.github.com/users/Natim/following{/other_user}","gists_url":"https://api.github.com/users/Natim/gists{/gist_id}","starred_url":"https://api.github.com/users/Natim/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Natim/subscriptions","organizations_url":"https://api.github.com/users/Natim/orgs","repos_url":"https://api.github.com/users/Natim/repos","events_url":"https://api.github.com/users/Natim/events{/privacy}","received_events_url":"https://api.github.com/users/Natim/received_events","type":"User","site_admin":false},"body":"- [x] Document installation #8\r\n- [x] Document json2kinto importation command #18 \r\n- [x] Document kinto2xml exportation command #19\r\n","created_at":"2016-05-11T09:15:09Z","updated_at":"2016-05-11T09:29:10Z","closed_at":null,"merged_at":null,"merge_commit_sha":"efa6ed341f60eda7e53cc4e14dd770a507f49e75","assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/mozilla-services/amo2kinto/pulls/23/commits","review_comments_url":"https://api.github.com/repos/mozilla-services/amo2kinto/pulls/23/comments","review_comment_url":"https://api.github.com/repos/mozilla-services/amo2kinto/pulls/comments{/number}","comments_url":"https://api.github.com/repos/mozilla-services/amo2kinto/issues/23/comments","statuses_url":"https://api.github.com/repos/mozilla-services/amo2kinto/statuses/7c0d8dad7955c7878188b096b69f13aa626b0ef4","head":{"label":"mozilla-services:document-installation","ref":"document-installation","sha":"7c0d8dad7955c7878188b096b69f13aa626b0ef4","user":{"login":"mozilla-services","id":1066228,"avatar_url":"https://avatars.githubusercontent.com/u/1066228?v=3","gravatar_id":"","url":"https://api.github.com/users/mozilla-services","html_url":"https://github.com/mozilla-services","followers_url":"https://api.github.com/users/mozilla-services/followers","following_url":"https://api.github.com/users/mozilla-services/following{/other_user}","gists_url":"https://api.github.com/users/mozilla-services/gists{/gist_id}","starred_url":"https://api.github.com/users/mozilla-services/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mozilla-services/subscriptions","organizations_url":"https://api.github.com/users/mozilla-services/orgs","repos_url":"https://api.github.com/users/mozilla-services/repos","events_url":"https://api.github.com/users/mozilla-services/events{/privacy}","received_events_url":"https://api.github.com/users/mozilla-services/received_events","type":"Organization","site_admin":false},"repo":{"id":51925233,"name":"amo2kinto","full_name":"mozilla-services/amo2kinto","owner":{"login":"mozilla-services","id":1066228,"avatar_url":"https://avatars.githubusercontent.com/u/1066228?v=3","gravatar_id":"","url":"https://api.github.com/users/mozilla-services","html_url":"https://github.com/mozilla-services","followers_url":"https://api.github.com/users/mozilla-services/followers","following_url":"https://api.github.com/users/mozilla-services/following{/other_user}","gists_url":"https://api.github.com/users/mozilla-services/gists{/gist_id}","starred_url":"https://api.github.com/users/mozilla-services/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mozilla-services/subscriptions","organizations_url":"https://api.github.com/users/mozilla-services/orgs","repos_url":"https://api.github.com/users/mozilla-services/repos","events_url":"https://api.github.com/users/mozilla-services/events{/privacy}","received_events_url":"https://api.github.com/users/mozilla-services/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/mozilla-services/amo2kinto","description":"Generate the blocklists.xml file from Kinto collections.","fork":false,"url":"https://api.github.com/repos/mozilla-services/amo2kinto","forks_url":"https://api.github.com/repos/mozilla-services/amo2kinto/forks","keys_url":"https://api.github.com/repos/mozilla-services/amo2kinto/keys{/key_id}","collaborators_url":"https://api.github.com/repos/mozilla-services/amo2kinto/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/mozilla-services/amo2kinto/teams","hooks_url":"https://api.github.com/repos/mozilla-services/amo2kinto/hooks","issue_events_url":"https://api.github.com/repos/mozilla-services/amo2kinto/issues/events{/number}","events_url":"https://api.github.com/repos/mozilla-services/amo2kinto/events","assignees_url":"https://api.github.com/repos/mozilla-services/amo2kinto/assignees{/user}","branches_url":"https://api.github.com/repos/mozilla-services/amo2kinto/branches{/branch}","tags_url":"https://api.github.com/repos/mozilla-services/amo2kinto/tags","blobs_url":"https://api.github.com/repos/mozilla-services/amo2kinto/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/mozilla-services/amo2kinto/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/mozilla-services/amo2kinto/git/refs{/sha}","trees_url":"https://api.github.com/repos/mozilla-services/amo2kinto/git/trees{/sha}","statuses_url":"https://api.github.com/repos/mozilla-services/amo2kinto/statuses/{sha}","languages_url":"https://api.github.com/repos/mozilla-services/amo2kinto/languages","stargazers_url":"https://api.github.com/repos/mozilla-services/amo2kinto/stargazers","contributors_url":"https://api.github.com/repos/mozilla-services/amo2kinto/contributors","subscribers_url":"https://api.github.com/repos/mozilla-services/amo2kinto/subscribers","subscription_url":"https://api.github.com/repos/mozilla-services/amo2kinto/subscription","commits_url":"https://api.github.com/repos/mozilla-services/amo2kinto/commits{/sha}","git_commits_url":"https://api.github.com/repos/mozilla-services/amo2kinto/git/commits{/sha}","comments_url":"https://api.github.com/repos/mozilla-services/amo2kinto/comments{/number}","issue_comment_url":"https://api.github.com/repos/mozilla-services/amo2kinto/issues/comments{/number}","contents_url":"https://api.github.com/repos/mozilla-services/amo2kinto/contents/{+path}","compare_url":"https://api.github.com/repos/mozilla-services/amo2kinto/compare/{base}...{head}","merges_url":"https://api.github.com/repos/mozilla-services/amo2kinto/merges","archive_url":"https://api.github.com/repos/mozilla-services/amo2kinto/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/mozilla-services/amo2kinto/downloads","issues_url":"https://api.github.com/repos/mozilla-services/amo2kinto/issues{/number}","pulls_url":"https://api.github.com/repos/mozilla-services/amo2kinto/pulls{/number}","milestones_url":"https://api.github.com/repos/mozilla-services/amo2kinto/milestones{/number}","notifications_url":"https://api.github.com/repos/mozilla-services/amo2kinto/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/mozilla-services/amo2kinto/labels{/name}","releases_url":"https://api.github.com/repos/mozilla-services/amo2kinto/releases{/id}","deployments_url":"https://api.github.com/repos/mozilla-services/amo2kinto/deployments","created_at":"2016-02-17T13:40:35Z","updated_at":"2016-05-11T08:45:44Z","pushed_at":"2016-05-11T09:29:09Z","git_url":"git://github.com/mozilla-services/amo2kinto.git","ssh_url":"git@github.com:mozilla-services/amo2kinto.git","clone_url":"https://github.com/mozilla-services/amo2kinto.git","svn_url":"https://github.com/mozilla-services/amo2kinto","homepage":null,"size":185,"stargazers_count":0,"watchers_count":0,"language":"Python","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":4,"forks":0,"open_issues":4,"watchers":0,"default_branch":"master"}},"base":{"label":"mozilla-services:master","ref":"master","sha":"0d28a771174a5d76a334109e0fbdb9316c4c396d","user":{"login":"mozilla-services","id":1066228,"avatar_url":"https://avatars.githubusercontent.com/u/1066228?v=3","gravatar_id":"","url":"https://api.github.com/users/mozilla-services","html_url":"https://github.com/mozilla-services","followers_url":"https://api.github.com/users/mozilla-services/followers","following_url":"https://api.github.com/users/mozilla-services/following{/other_user}","gists_url":"https://api.github.com/users/mozilla-services/gists{/gist_id}","starred_url":"https://api.github.com/users/mozilla-services/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mozilla-services/subscriptions","organizations_url":"https://api.github.com/users/mozilla-services/orgs","repos_url":"https://api.github.com/users/mozilla-services/repos","events_url":"https://api.github.com/users/mozilla-services/events{/privacy}","received_events_url":"https://api.github.com/users/mozilla-services/received_events","type":"Organization","site_admin":false},"repo":{"id":51925233,"name":"amo2kinto","full_name":"mozilla-services/amo2kinto","owner":{"login":"mozilla-services","id":1066228,"avatar_url":"https://avatars.githubusercontent.com/u/1066228?v=3","gravatar_id":"","url":"https://api.github.com/users/mozilla-services","html_url":"https://github.com/mozilla-services","followers_url":"https://api.github.com/users/mozilla-services/followers","following_url":"https://api.github.com/users/mozilla-services/following{/other_user}","gists_url":"https://api.github.com/users/mozilla-services/gists{/gist_id}","starred_url":"https://api.github.com/users/mozilla-services/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mozilla-services/subscriptions","organizations_url":"https://api.github.com/users/mozilla-services/orgs","repos_url":"https://api.github.com/users/mozilla-services/repos","events_url":"https://api.github.com/users/mozilla-services/events{/privacy}","received_events_url":"https://api.github.com/users/mozilla-services/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/mozilla-services/amo2kinto","description":"Generate the blocklists.xml file from Kinto collections.","fork":false,"url":"https://api.github.com/repos/mozilla-services/amo2kinto","forks_url":"https://api.github.com/repos/mozilla-services/amo2kinto/forks","keys_url":"https://api.github.com/repos/mozilla-services/amo2kinto/keys{/key_id}","collaborators_url":"https://api.github.com/repos/mozilla-services/amo2kinto/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/mozilla-services/amo2kinto/teams","hooks_url":"https://api.github.com/repos/mozilla-services/amo2kinto/hooks","issue_events_url":"https://api.github.com/repos/mozilla-services/amo2kinto/issues/events{/number}","events_url":"https://api.github.com/repos/mozilla-services/amo2kinto/events","assignees_url":"https://api.github.com/repos/mozilla-services/amo2kinto/assignees{/user}","branches_url":"https://api.github.com/repos/mozilla-services/amo2kinto/branches{/branch}","tags_url":"https://api.github.com/repos/mozilla-services/amo2kinto/tags","blobs_url":"https://api.github.com/repos/mozilla-services/amo2kinto/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/mozilla-services/amo2kinto/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/mozilla-services/amo2kinto/git/refs{/sha}","trees_url":"https://api.github.com/repos/mozilla-services/amo2kinto/git/trees{/sha}","statuses_url":"https://api.github.com/repos/mozilla-services/amo2kinto/statuses/{sha}","languages_url":"https://api.github.com/repos/mozilla-services/amo2kinto/languages","stargazers_url":"https://api.github.com/repos/mozilla-services/amo2kinto/stargazers","contributors_url":"https://api.github.com/repos/mozilla-services/amo2kinto/contributors","subscribers_url":"https://api.github.com/repos/mozilla-services/amo2kinto/subscribers","subscription_url":"https://api.github.com/repos/mozilla-services/amo2kinto/subscription","commits_url":"https://api.github.com/repos/mozilla-services/amo2kinto/commits{/sha}","git_commits_url":"https://api.github.com/repos/mozilla-services/amo2kinto/git/commits{/sha}","comments_url":"https://api.github.com/repos/mozilla-services/amo2kinto/comments{/number}","issue_comment_url":"https://api.github.com/repos/mozilla-services/amo2kinto/issues/comments{/number}","contents_url":"https://api.github.com/repos/mozilla-services/amo2kinto/contents/{+path}","compare_url":"https://api.github.com/repos/mozilla-services/amo2kinto/compare/{base}...{head}","merges_url":"https://api.github.com/repos/mozilla-services/amo2kinto/merges","archive_url":"https://api.github.com/repos/mozilla-services/amo2kinto/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/mozilla-services/amo2kinto/downloads","issues_url":"https://api.github.com/repos/mozilla-services/amo2kinto/issues{/number}","pulls_url":"https://api.github.com/repos/mozilla-services/amo2kinto/pulls{/number}","milestones_url":"https://api.github.com/repos/mozilla-services/amo2kinto/milestones{/number}","notifications_url":"https://api.github.com/repos/mozilla-services/amo2kinto/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/mozilla-services/amo2kinto/labels{/name}","releases_url":"https://api.github.com/repos/mozilla-services/amo2kinto/releases{/id}","deployments_url":"https://api.github.com/repos/mozilla-services/amo2kinto/deployments","created_at":"2016-02-17T13:40:35Z","updated_at":"2016-05-11T08:45:44Z","pushed_at":"2016-05-11T09:29:09Z","git_url":"git://github.com/mozilla-services/amo2kinto.git","ssh_url":"git@github.com:mozilla-services/amo2kinto.git","clone_url":"https://github.com/mozilla-services/amo2kinto.git","svn_url":"https://github.com/mozilla-services/amo2kinto","homepage":null,"size":185,"stargazers_count":0,"watchers_count":0,"language":"Python","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":4,"forks":0,"open_issues":4,"watchers":0,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/mozilla-services/amo2kinto/pulls/23"},"html":{"href":"https://github.com/mozilla-services/amo2kinto/pull/23"},"issue":{"href":"https://api.github.com/repos/mozilla-services/amo2kinto/issues/23"},"comments":{"href":"https://api.github.com/repos/mozilla-services/amo2kinto/issues/23/comments"},"review_comments":{"href":"https://api.github.com/repos/mozilla-services/amo2kinto/pulls/23/comments"},"review_comment":{"href":"https://api.github.com/repos/mozilla-services/amo2kinto/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/mozilla-services/amo2kinto/pulls/23/commits"},"statuses":{"href":"https://api.github.com/repos/mozilla-services/amo2kinto/statuses/7c0d8dad7955c7878188b096b69f13aa626b0ef4"}}}}