{"action":"created","comment":{"url":"https://api.github.com/repos/edx/configuration/pulls/comments/54234164","id":54234164,"diff_hunk":"@@ -0,0 +1,139 @@\n+Jenkins Analytics\n+=================\n+\n+A role that sets-up Jenkins for deploying analytics stack\n+on AWS EMR.\n+\n+It does following steps:\n+\n+* Installs jenkins using ``jenkins_master``\n+* Configures ``config.xml`` to enable security and use\n+  Linux Auth Domain.\n+* Enable the use of jenkins clis\n+\n+Configuration\n+-------------\n+\n+Following variables are used by this role:\n+\n+You need to provide a json file containing credentials\n+to be uploaded to server. This file is **a local file**\n+suitable for use in ``copy`` module. This file is set by\n+following variable:\n+\n+    jenkins_credentials_file:  files/ansible-jenkins-credentials.json\n+\n+\n+Used by command waiting on jenkins start-up after running ``jenkins_master``\n+role:\n+\n+    jenkins_connection_retries: 60\n+    jenkins_connection_delay: 0.5\n+\n+Jenkins auth realm, that is a method used by jenkins to authenticate users.\n+Realm type stored in ``jenkins_auth_realm.name`` variable.\n+\n+In future we will try enable other auth domains, while\n+preserving ability to run cli.\n+\n+### Unix Realm\n+\n+For now only ``unix`` realm supported --- which requires every user to have a\n+shell account on the server.\n+\n+Unix realm requires following settings:\n+\n+``service`` --- jenkins uses PAM configuration for this service. `su` is\n+a safe choice as it doesn't require that user has ability to login remotely/\n+\n+``plain_password`` --- plaintext password, **you should change** default values\n+\n+``crypted_password`` --- encrypted password, to obtain it use ``mkpasswd``\n+command, for example: `` mkpasswd --method=sha-512``\n+\n+Example realm configuration:\n+\n+    jenkins_auth_realm:\n+      name: unix\n+      service: su\n+      plain_password: jenkins\n+      crypted_password: $6$rAVyI.p2wXVDKk5w$y0G1MQehmHtvaPgdtbrnvAsBqYQ99g939vxrdLXtPQCh/e7GJVwbnqIKZpve8EcMLTtq.7sZwTBYV9Tdjgf1k.\n+\n+\n+### Seed job configuration\n+\n+Seed job is configured in ``jenkins_seed_job`` variable, which has following\n+attributes:\n+\n+``name`` Name of the job in Jenkins.\n+\n+``time_trigger``: A jenkins cron entry defining how often this job is ran.\n+\n+``removed_job_action`` what to do when job created by the seed job is deleted.\n+This can be either  ``DELETE`` or``IGNORE``.\n+``removed_view_action`` What to do when view created by the seed job is removed.\n+This can be either  ``DELETE`` or``IGNORE``.\n+``scm`` Scm object is used to define from where to download\n+the jobs. It has following properties:\n+\n+``scm.type`` For today it must have value of ``git``.\n+\n+``scm.url`` Url for the repository\n+\n+``scm.credential_id`` Id of credential used to authenticate to the repository.\n+\n+``scm.targed_jobs`` A shell glob expression relative to repo selecting jobs\n+to import\n+\n+``scm.additional_classpath`` A path relative to repo root, pointing to a path\n+that contains additional groovy scripts used by the seed jons.\n+\n+Example scm configuration:\n+\n+    jenkins_seed_job:\n+      name: seed\n+      time_trigger: \"H * * * *\"\n+      removed_job_action: \"DELETE\"\n+      removed_view_action: \"IGNORE\"\n+      scm:\n+        type: git\n+        url: \"git@github.com:edx-ops/edx-jenkins-job-dsl.git\"\n+        credential_id: \"github-deploy-key\"\n+        targed_jobs: \"jobs/analytics-edx-jenkins.edx.org/*Jobs.groovy\"\n+        additional_classpath: \"src/main/groovy\"\n+\n+Credential file example\n+-----------------------\n+\n+To generate a proper json credential file I strongly suggest:\n+\n+1. Write the file in ``yaml`` format.\n+2. Convert it by using for example:\n+\n+        cat file.yaml |  python -c \"import yaml,json,sys; print(json.dumps(yaml.load(sys.stdin)))\" > file.json\n+\n+\n+See ``jenkins_analytics/meta/ansible-jenkins-credentials.yaml`` a example\n+configuration file format.\n+\n+Known issues\n+------------\n+\n+1. SSH keys without password don't work.\n+2. This playbook should have been a module: ``_execute_ansible_cli.yaml``\n+3. Anonymous user has discover and get job permission, as without it\n+  ``get-job``, ``build <<job>>`` commands wouldn't work.\n+  Giving anonymous these permissions a workaround for\n+  transient Jenkins issue (reported [couple][1] [of][2] [times][3]).\n+4. We force unix authentication method --- that is every user that can login","path":"playbooks/roles/jenkins_analytics/README.md","position":128,"original_position":128,"commit_id":"0081d4de25a0d9139c7891279ed834a1b90d0e32","original_commit_id":"1450c0710a4434fc77bc8a68597006f304b506d8","user":{"login":"itsjeyd","id":961441,"avatar_url":"https://avatars.githubusercontent.com/u/961441?v=3","gravatar_id":"","url":"https://api.github.com/users/itsjeyd","html_url":"https://github.com/itsjeyd","followers_url":"https://api.github.com/users/itsjeyd/followers","following_url":"https://api.github.com/users/itsjeyd/following{/other_user}","gists_url":"https://api.github.com/users/itsjeyd/gists{/gist_id}","starred_url":"https://api.github.com/users/itsjeyd/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/itsjeyd/subscriptions","organizations_url":"https://api.github.com/users/itsjeyd/orgs","repos_url":"https://api.github.com/users/itsjeyd/repos","events_url":"https://api.github.com/users/itsjeyd/events{/privacy}","received_events_url":"https://api.github.com/users/itsjeyd/received_events","type":"User","site_admin":false},"body":"@jbzdak \"---\" ⟶ \"--\"","created_at":"2016-02-26T11:49:02Z","updated_at":"2016-02-26T11:49:02Z","html_url":"https://github.com/edx/configuration/pull/2810#discussion_r54234164","pull_request_url":"https://api.github.com/repos/edx/configuration/pulls/2810","_links":{"self":{"href":"https://api.github.com/repos/edx/configuration/pulls/comments/54234164"},"html":{"href":"https://github.com/edx/configuration/pull/2810#discussion_r54234164"},"pull_request":{"href":"https://api.github.com/repos/edx/configuration/pulls/2810"}}},"pull_request":{"url":"https://api.github.com/repos/edx/configuration/pulls/2810","id":60662663,"html_url":"https://github.com/edx/configuration/pull/2810","diff_url":"https://github.com/edx/configuration/pull/2810.diff","patch_url":"https://github.com/edx/configuration/pull/2810.patch","issue_url":"https://api.github.com/repos/edx/configuration/issues/2810","number":2810,"state":"open","locked":false,"title":"WIP OLIVE-19 Enable automatic provisioning of jenkins instance dedicated to running analytics tasks. ","user":{"login":"jbzdak","id":112872,"avatar_url":"https://avatars.githubusercontent.com/u/112872?v=3","gravatar_id":"","url":"https://api.github.com/users/jbzdak","html_url":"https://github.com/jbzdak","followers_url":"https://api.github.com/users/jbzdak/followers","following_url":"https://api.github.com/users/jbzdak/following{/other_user}","gists_url":"https://api.github.com/users/jbzdak/gists{/gist_id}","starred_url":"https://api.github.com/users/jbzdak/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jbzdak/subscriptions","organizations_url":"https://api.github.com/users/jbzdak/orgs","repos_url":"https://api.github.com/users/jbzdak/repos","events_url":"https://api.github.com/users/jbzdak/events{/privacy}","received_events_url":"https://api.github.com/users/jbzdak/received_events","type":"User","site_admin":false},"body":"This is a WIP PR, posted mostly to check if Travis will give me suprises :) \r\n\r\n# Testing instructions \r\n\r\n0. Verify that you have access to: https://github.com/edx-ops/edx-jenkins-job-dsl/\r\n1. Generate a new ssh-key with a password. **If you won't provide password it will not work**. \r\n2. Attach this ssh-key to your github account. \r\n3. Create a ``yaml`` file in a place that will be searched by ansible commands (for example in ``playbooks`` folder). This file will contain credentials that will be loaded by the jenkins (i'll post example at the end of instructions)\r\n4. Go to ``vagrant/base/analytic_jenkins`` and issue ``vagrant up``\r\n5. Expected result is that you will get a working jenkins istance that you'll be able to log in to ``192.168.33.15:8080`` or ``localhost:8080``. \r\n6. You'll be able to login using ``jenkins:jenkins`` credentials to the instance. \r\n7. There will be a role called ``seed`` which role was already ran single time with success result.\r\n8. During the run ``seed`` role created a new role called: ``stub-analytics-jobs``. \r\n9. Remove the key you added to github in step 2. :) \r\n\r\n# Documentation check \r\n\r\n1. Check documentation of the new role ``playbooks/roles/jenkins_analytics/README.md`` to check whethet you understand how to configure the this role. \r\n2. Check outstanding issues to check which ones are grevious enough to fix right now :) \r\n\r\n# Credentials file\r\n\r\nCredentials file for now is a json file created from yaml with this schema, you'll have to actually use ``id`` equal to ``github-deploy-key``, or it must just match ``jenkins_seed_job.scm.credential_id``. \r\n\r\n   ```\r\ncredentials:\r\n      - id: github-deploy-key\r\n        scope: GLOBAL\r\n        username: your username\r\n        type: ssh-private-key\r\n        passphrase: 'foobar'\r\n        description: Autogenerated by ansible\r\n        privatekey: |\r\n          -----BEGIN RSA PRIVATE KEY-----\r\n          Proc-Type: 4,ENCRYPTED\r\n          DEK-Info: AES-128-CBC,...\r\n    \r\n        \r\n          -----END RSA PRIVATE KEY-----\r\n```\r\n","created_at":"2016-02-25T11:17:54Z","updated_at":"2016-02-26T11:49:02Z","closed_at":null,"merged_at":null,"merge_commit_sha":"bb971fc039c96fdeabc518447d4201c61fabc0f5","assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/edx/configuration/pulls/2810/commits","review_comments_url":"https://api.github.com/repos/edx/configuration/pulls/2810/comments","review_comment_url":"https://api.github.com/repos/edx/configuration/pulls/comments{/number}","comments_url":"https://api.github.com/repos/edx/configuration/issues/2810/comments","statuses_url":"https://api.github.com/repos/edx/configuration/statuses/0081d4de25a0d9139c7891279ed834a1b90d0e32","head":{"label":"edx:jbzdak/olive-19-auto-provision-analytics-devstack","ref":"jbzdak/olive-19-auto-provision-analytics-devstack","sha":"0081d4de25a0d9139c7891279ed834a1b90d0e32","user":{"login":"edx","id":3179841,"avatar_url":"https://avatars.githubusercontent.com/u/3179841?v=3","gravatar_id":"","url":"https://api.github.com/users/edx","html_url":"https://github.com/edx","followers_url":"https://api.github.com/users/edx/followers","following_url":"https://api.github.com/users/edx/following{/other_user}","gists_url":"https://api.github.com/users/edx/gists{/gist_id}","starred_url":"https://api.github.com/users/edx/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/edx/subscriptions","organizations_url":"https://api.github.com/users/edx/orgs","repos_url":"https://api.github.com/users/edx/repos","events_url":"https://api.github.com/users/edx/events{/privacy}","received_events_url":"https://api.github.com/users/edx/received_events","type":"Organization","site_admin":false},"repo":{"id":9478219,"name":"configuration","full_name":"edx/configuration","owner":{"login":"edx","id":3179841,"avatar_url":"https://avatars.githubusercontent.com/u/3179841?v=3","gravatar_id":"","url":"https://api.github.com/users/edx","html_url":"https://github.com/edx","followers_url":"https://api.github.com/users/edx/followers","following_url":"https://api.github.com/users/edx/following{/other_user}","gists_url":"https://api.github.com/users/edx/gists{/gist_id}","starred_url":"https://api.github.com/users/edx/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/edx/subscriptions","organizations_url":"https://api.github.com/users/edx/orgs","repos_url":"https://api.github.com/users/edx/repos","events_url":"https://api.github.com/users/edx/events{/privacy}","received_events_url":"https://api.github.com/users/edx/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/edx/configuration","description":"a simple, but flexible, way for anyone to stand up an instance of the edX platform that is fully configured and ready-to-go","fork":false,"url":"https://api.github.com/repos/edx/configuration","forks_url":"https://api.github.com/repos/edx/configuration/forks","keys_url":"https://api.github.com/repos/edx/configuration/keys{/key_id}","collaborators_url":"https://api.github.com/repos/edx/configuration/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/edx/configuration/teams","hooks_url":"https://api.github.com/repos/edx/configuration/hooks","issue_events_url":"https://api.github.com/repos/edx/configuration/issues/events{/number}","events_url":"https://api.github.com/repos/edx/configuration/events","assignees_url":"https://api.github.com/repos/edx/configuration/assignees{/user}","branches_url":"https://api.github.com/repos/edx/configuration/branches{/branch}","tags_url":"https://api.github.com/repos/edx/configuration/tags","blobs_url":"https://api.github.com/repos/edx/configuration/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/edx/configuration/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/edx/configuration/git/refs{/sha}","trees_url":"https://api.github.com/repos/edx/configuration/git/trees{/sha}","statuses_url":"https://api.github.com/repos/edx/configuration/statuses/{sha}","languages_url":"https://api.github.com/repos/edx/configuration/languages","stargazers_url":"https://api.github.com/repos/edx/configuration/stargazers","contributors_url":"https://api.github.com/repos/edx/configuration/contributors","subscribers_url":"https://api.github.com/repos/edx/configuration/subscribers","subscription_url":"https://api.github.com/repos/edx/configuration/subscription","commits_url":"https://api.github.com/repos/edx/configuration/commits{/sha}","git_commits_url":"https://api.github.com/repos/edx/configuration/git/commits{/sha}","comments_url":"https://api.github.com/repos/edx/configuration/comments{/number}","issue_comment_url":"https://api.github.com/repos/edx/configuration/issues/comments{/number}","contents_url":"https://api.github.com/repos/edx/configuration/contents/{+path}","compare_url":"https://api.github.com/repos/edx/configuration/compare/{base}...{head}","merges_url":"https://api.github.com/repos/edx/configuration/merges","archive_url":"https://api.github.com/repos/edx/configuration/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/edx/configuration/downloads","issues_url":"https://api.github.com/repos/edx/configuration/issues{/number}","pulls_url":"https://api.github.com/repos/edx/configuration/pulls{/number}","milestones_url":"https://api.github.com/repos/edx/configuration/milestones{/number}","notifications_url":"https://api.github.com/repos/edx/configuration/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/edx/configuration/labels{/name}","releases_url":"https://api.github.com/repos/edx/configuration/releases{/id}","deployments_url":"https://api.github.com/repos/edx/configuration/deployments","created_at":"2013-04-16T17:23:42Z","updated_at":"2016-02-24T03:40:22Z","pushed_at":"2016-02-26T11:29:12Z","git_url":"git://github.com/edx/configuration.git","ssh_url":"git@github.com:edx/configuration.git","clone_url":"https://github.com/edx/configuration.git","svn_url":"https://github.com/edx/configuration","homepage":"","size":10852,"stargazers_count":496,"watchers_count":496,"language":"Python","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":457,"mirror_url":null,"open_issues_count":124,"forks":457,"open_issues":124,"watchers":496,"default_branch":"master"}},"base":{"label":"edx:master","ref":"master","sha":"8f03d8611bd93c6ba3627c14a6a0b14d2f8c3828","user":{"login":"edx","id":3179841,"avatar_url":"https://avatars.githubusercontent.com/u/3179841?v=3","gravatar_id":"","url":"https://api.github.com/users/edx","html_url":"https://github.com/edx","followers_url":"https://api.github.com/users/edx/followers","following_url":"https://api.github.com/users/edx/following{/other_user}","gists_url":"https://api.github.com/users/edx/gists{/gist_id}","starred_url":"https://api.github.com/users/edx/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/edx/subscriptions","organizations_url":"https://api.github.com/users/edx/orgs","repos_url":"https://api.github.com/users/edx/repos","events_url":"https://api.github.com/users/edx/events{/privacy}","received_events_url":"https://api.github.com/users/edx/received_events","type":"Organization","site_admin":false},"repo":{"id":9478219,"name":"configuration","full_name":"edx/configuration","owner":{"login":"edx","id":3179841,"avatar_url":"https://avatars.githubusercontent.com/u/3179841?v=3","gravatar_id":"","url":"https://api.github.com/users/edx","html_url":"https://github.com/edx","followers_url":"https://api.github.com/users/edx/followers","following_url":"https://api.github.com/users/edx/following{/other_user}","gists_url":"https://api.github.com/users/edx/gists{/gist_id}","starred_url":"https://api.github.com/users/edx/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/edx/subscriptions","organizations_url":"https://api.github.com/users/edx/orgs","repos_url":"https://api.github.com/users/edx/repos","events_url":"https://api.github.com/users/edx/events{/privacy}","received_events_url":"https://api.github.com/users/edx/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/edx/configuration","description":"a simple, but flexible, way for anyone to stand up an instance of the edX platform that is fully configured and ready-to-go","fork":false,"url":"https://api.github.com/repos/edx/configuration","forks_url":"https://api.github.com/repos/edx/configuration/forks","keys_url":"https://api.github.com/repos/edx/configuration/keys{/key_id}","collaborators_url":"https://api.github.com/repos/edx/configuration/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/edx/configuration/teams","hooks_url":"https://api.github.com/repos/edx/configuration/hooks","issue_events_url":"https://api.github.com/repos/edx/configuration/issues/events{/number}","events_url":"https://api.github.com/repos/edx/configuration/events","assignees_url":"https://api.github.com/repos/edx/configuration/assignees{/user}","branches_url":"https://api.github.com/repos/edx/configuration/branches{/branch}","tags_url":"https://api.github.com/repos/edx/configuration/tags","blobs_url":"https://api.github.com/repos/edx/configuration/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/edx/configuration/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/edx/configuration/git/refs{/sha}","trees_url":"https://api.github.com/repos/edx/configuration/git/trees{/sha}","statuses_url":"https://api.github.com/repos/edx/configuration/statuses/{sha}","languages_url":"https://api.github.com/repos/edx/configuration/languages","stargazers_url":"https://api.github.com/repos/edx/configuration/stargazers","contributors_url":"https://api.github.com/repos/edx/configuration/contributors","subscribers_url":"https://api.github.com/repos/edx/configuration/subscribers","subscription_url":"https://api.github.com/repos/edx/configuration/subscription","commits_url":"https://api.github.com/repos/edx/configuration/commits{/sha}","git_commits_url":"https://api.github.com/repos/edx/configuration/git/commits{/sha}","comments_url":"https://api.github.com/repos/edx/configuration/comments{/number}","issue_comment_url":"https://api.github.com/repos/edx/configuration/issues/comments{/number}","contents_url":"https://api.github.com/repos/edx/configuration/contents/{+path}","compare_url":"https://api.github.com/repos/edx/configuration/compare/{base}...{head}","merges_url":"https://api.github.com/repos/edx/configuration/merges","archive_url":"https://api.github.com/repos/edx/configuration/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/edx/configuration/downloads","issues_url":"https://api.github.com/repos/edx/configuration/issues{/number}","pulls_url":"https://api.github.com/repos/edx/configuration/pulls{/number}","milestones_url":"https://api.github.com/repos/edx/configuration/milestones{/number}","notifications_url":"https://api.github.com/repos/edx/configuration/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/edx/configuration/labels{/name}","releases_url":"https://api.github.com/repos/edx/configuration/releases{/id}","deployments_url":"https://api.github.com/repos/edx/configuration/deployments","created_at":"2013-04-16T17:23:42Z","updated_at":"2016-02-24T03:40:22Z","pushed_at":"2016-02-26T11:29:12Z","git_url":"git://github.com/edx/configuration.git","ssh_url":"git@github.com:edx/configuration.git","clone_url":"https://github.com/edx/configuration.git","svn_url":"https://github.com/edx/configuration","homepage":"","size":10852,"stargazers_count":496,"watchers_count":496,"language":"Python","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":457,"mirror_url":null,"open_issues_count":124,"forks":457,"open_issues":124,"watchers":496,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/edx/configuration/pulls/2810"},"html":{"href":"https://github.com/edx/configuration/pull/2810"},"issue":{"href":"https://api.github.com/repos/edx/configuration/issues/2810"},"comments":{"href":"https://api.github.com/repos/edx/configuration/issues/2810/comments"},"review_comments":{"href":"https://api.github.com/repos/edx/configuration/pulls/2810/comments"},"review_comment":{"href":"https://api.github.com/repos/edx/configuration/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/edx/configuration/pulls/2810/commits"},"statuses":{"href":"https://api.github.com/repos/edx/configuration/statuses/0081d4de25a0d9139c7891279ed834a1b90d0e32"}}}}