{"action":"created","comment":{"url":"https://api.github.com/repos/edx/configuration/pulls/comments/56707514","id":56707514,"diff_hunk":"@@ -0,0 +1,202 @@\n+# Jenkins Analytics\n+\n+A role that sets up Jenkins for scheduling analytics tasks. \n+\n+This role performs the following steps:\n+\n+* Installs Jenkins using `jenkins_master`.\n+* Configures `config.xml` to enable security and use\n+  Linux Auth Domain.\n+* Creates Jenkins credentials. \n+* Enables the use of Jenkins CLI. \n+* Installs a seed job from configured repository, launches it and waits\n+  for it to finish.\n+\n+## Configuration\n+\n+When you are using vagrant you **need** to set `VAGRANT_JENKINS_LOCAL_VARS_FILE`\n+environment variable. This variable must point to a file containing \n+all required variables from this section.\n+\n+This file needs to contain, at least, the following variables \n+(see the next few sections for more information about them): \n+\n+* `JENKINS_ANALYTICS_USER_PASSWORD_HASHED` \n+* `JENKINS_ANALYTICS_USER_PASSWORD_PLAIN`\n+* `JENKINS_ANALYTICS_GITHUB_KEY` or `JENKINS_ANALYTICS_CREDENTIALS`\n+\n+ \n+### End-user editable configuration \n+\n+#### Jenkins user password\n+\n+You'll need to override default jenkins user password, please do that\n+as this sets up the **shell** password for this user. \n+\n+You'll need to set both a plain password and a hashed one.\n+To obtain hashed password use `mkpasswd` command, for example: \n+`mkpasswd --method=sha-512`. (Note: hashed password is required \n+to have clean \"changed\"/\"unchanged\" notification for this step \n+in Ansible.) \n+\n+* `JENKINS_ANALYTICS_USER_PASSWORD_HASHED`: hashed password \n+* `JENKINS_ANALYTICS_USER_PASSWORD_PLAIN`: plain password\n+\n+#### Jenkins seed job configuration \n+\n+This will be filled as part of PR[#2830](https://github.com/edx/configuration/pull/2830). \n+For now go with defaults. \n+\n+#### Jenkins credentials\n+\n+Jenkins contains its own credential store, to fill it with credentials \n+please use `JENKINS_ANALYTICS_CREDENTIALS` variable. This variable \n+is a list of objects, each object represents a single credential,\n+for now passwords and ssh-keys are supported. \n+\n+If you only need credentials to access github repositories\n+you can override `JENKINS_ANALYTICS_GITHUB_KEY`,\n+which should contain contents of private key used for \n+authentication to checkout github repositories.  \n+\n+Each credential has an unique id, this credential id is used by \n+Jenkins jobs to match a task requiring credential with a \n+credential for given task. \n+\n+Examples of credentials variables:\n+ \n+    JENKINS_ANALYTICS_GITHUB_KEY: \"{{ lookup('file', 'path to keyfile') }}\" \n+        \n+    JENKINS_ANALYTICS_CREDENTIALS:\n+      # id is a scope-unique credential identifier\n+      - id: test-password\n+        # Scope must be global, to have other scopes you'll need ot modify addCredentials.groovy\n+        scope: GLOBAL\n+        # Username associated with this password\n+        username: jenkins\n+        type: username-password\n+        description: Autogenerated by ansible\n+        password: 'foobarbaz'\n+      # id is a scope-unique credential identifier\n+      - id: github-deploy-key\n+        scope: GLOBAL\n+        # Username this ssh-key is attached to\n+        username: git\n+        # Type of credential, see other entries for example\n+        type: ssh-private-key        \n+        passphrase: 'foobar'\n+        description: Generated by ansible\n+        privatekey: |\n+          -----BEGIN RSA PRIVATE KEY-----\n+          Proc-Type: 4,ENCRYPTED\n+          DEK-Info: AES-128-CBC,....\n+\n+          Key contents\n+          -----END RSA PRIVATE KEY-----\n+\n+#### Other useful variables\n+\n+* `JENKINS_ANALYTICS_CONCURRENT_JOBS_COUNT`: Configures number of \n+  executors (or concurrent jobs this Jenkins instance can \n+  execute). Defaults to `2`. \n+\n+### General configuration \n+\n+Following variables are used by this role:\n+\n+Variables used by command waiting on Jenkins start-up after running\n+`jenkins_master` role:\n+\n+    jenkins_connection_retries: 60\n+    jenkins_connection_delay: 0.5\n+\n+#### Auth realm\n+\n+Jenkins auth realm, that is a method used by jenkins to authenticate users.\n+Realm type stored in `jenkins_auth_realm.name` variable.\n+\n+In future we will try enable other auth domains, while\n+preserving ability to run cli.\n+\n+##### Unix Realm\n+\n+For now only `unix` realm supported -- which requires every user to have a\n+shell account on the server.\n+\n+Unix realm requires following settings:\n+\n+* `service`: Jenkins uses PAM configuration for this service. `su` is\n+a safe choice as it doesn't require a user to have the ability to login\n+remotely.\n+* `plain_password`:  plaintext password, **you should change** default values.\n+* `hashed_password`: hashed password\n+\n+Example realm configuration:\n+\n+    jenkins_auth_realm:\n+      name: unix\n+      service: su\n+      plain_password: jenkins\n+      hashed_password: $6$rAVyI.p2wXVDKk5w$y0G1MQehmHtvaPgdtbrnvAsBqYQ99g939vxrdLXtPQCh/e7GJVwbnqIKZpve8EcMLTtq.7sZwTBYV9Tdjgf1k.\n+\n+\n+#### Seed job configuration\n+\n+Seed job is configured in `jenkins_seed_job` variable, which has following\n+attributes:\n+\n+* `name`:  Name of the job in Jenkins.\n+* `time_trigger`: A Jenkins cron entry defining how often this job is ran.\n+* `removed_job_action`: what to do when job created by the seed job is deleted.\n+  This can be either  `DELETE` or`IGNORE`.\n+* `removed_view_action`: What to do when view created by the seed job is removed.\n+  This can be either  `DELETE` or`IGNORE`.\n+* `scm`: Scm object is used to define seed job repository and related settings.\n+  It has following properties:\n+  * `scm.type`: It must have value of `git`.\n+  * `scm.url`: URL for the repository.\n+  * `scm.credential_id`: Id of credential used to authenticate to the repository.\n+    This setting is optional. If it is missing or falsy, credentials will be omitted. \n+    Please note that when you use ssh repository url, you'll need to set up key irregardless \n+    of whether repository is public or private (to establish an ssh connection\n+    you need a valid public key). \n+  * `scm.target_jobs`: A shell glob expression relative to repo selecting\n+    jobs to import.\n+  * `scm.additional_classpath`: A path relative to repo root, pointing to a\n+     directory that contains additional groovy scripts used by the seed jobs.\n+\n+Example scm configuration:\n+\n+    jenkins_seed_job:\n+      name: seed\n+      time_trigger: \"H * * * *\"\n+      removed_job_action: \"DELETE\"\n+      removed_view_action: \"IGNORE\"\n+      scm:\n+        type: git\n+        url: \"git@github.com:edx-ops/edx-jenkins-job-dsl.git\"","path":"playbooks/roles/jenkins_analytics/README.md","position":181,"original_position":177,"commit_id":"428cd59bae5173a740aff208a68d92d2ed5d7add","original_commit_id":"0f1d016fca957bace76541ee4ef3f9c27c0a58db","user":{"login":"brianhw","id":2534095,"avatar_url":"https://avatars.githubusercontent.com/u/2534095?v=3","gravatar_id":"","url":"https://api.github.com/users/brianhw","html_url":"https://github.com/brianhw","followers_url":"https://api.github.com/users/brianhw/followers","following_url":"https://api.github.com/users/brianhw/following{/other_user}","gists_url":"https://api.github.com/users/brianhw/gists{/gist_id}","starred_url":"https://api.github.com/users/brianhw/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/brianhw/subscriptions","organizations_url":"https://api.github.com/users/brianhw/orgs","repos_url":"https://api.github.com/users/brianhw/repos","events_url":"https://api.github.com/users/brianhw/events{/privacy}","received_events_url":"https://api.github.com/users/brianhw/received_events","type":"User","site_admin":false},"body":"I thought it would be cleaner to not make references to non-open-sourced repos from open-source repos.  I had assumed that edx-ops would never had open-sourced repos, and that if we open-sourced the job-dsl repo, that it would be in the edx account.  And that for this level, it would be more general to have \"git@github.com:<account>/<jenkins-job-dsl-repo>.git\".  But if we are planning to open-source this soon, then I'd be willing to leave in place. ","created_at":"2016-03-18T19:09:14Z","updated_at":"2016-03-18T19:09:14Z","html_url":"https://github.com/edx/configuration/pull/2810#discussion_r56707514","pull_request_url":"https://api.github.com/repos/edx/configuration/pulls/2810","_links":{"self":{"href":"https://api.github.com/repos/edx/configuration/pulls/comments/56707514"},"html":{"href":"https://github.com/edx/configuration/pull/2810#discussion_r56707514"},"pull_request":{"href":"https://api.github.com/repos/edx/configuration/pulls/2810"}}},"pull_request":{"url":"https://api.github.com/repos/edx/configuration/pulls/2810","id":60662663,"html_url":"https://github.com/edx/configuration/pull/2810","diff_url":"https://github.com/edx/configuration/pull/2810.diff","patch_url":"https://github.com/edx/configuration/pull/2810.patch","issue_url":"https://api.github.com/repos/edx/configuration/issues/2810","number":2810,"state":"open","locked":false,"title":"OLIVE-19 Enable automatic provisioning of jenkins instance dedicated to running analytics tasks. ","user":{"login":"jbzdak","id":112872,"avatar_url":"https://avatars.githubusercontent.com/u/112872?v=3","gravatar_id":"","url":"https://api.github.com/users/jbzdak","html_url":"https://github.com/jbzdak","followers_url":"https://api.github.com/users/jbzdak/followers","following_url":"https://api.github.com/users/jbzdak/following{/other_user}","gists_url":"https://api.github.com/users/jbzdak/gists{/gist_id}","starred_url":"https://api.github.com/users/jbzdak/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jbzdak/subscriptions","organizations_url":"https://api.github.com/users/jbzdak/orgs","repos_url":"https://api.github.com/users/jbzdak/repos","events_url":"https://api.github.com/users/jbzdak/events{/privacy}","received_events_url":"https://api.github.com/users/jbzdak/received_events","type":"User","site_admin":false},"body":"# Testing instructions \r\n\r\n00. Create a virtualenv and install ``requirements.txt`` there. \r\n0. Verify that you have access to: https://github.com/edx-ops/edx-jenkins-job-dsl/\r\n1. Generate a new ssh-key. \r\n2. Attach this ssh-key to your github account. \r\n3. Create a yaml file containing configuration, required configuration is explained in ``Readme.md`` (if it's not obvious what to put in the file it's a documentation bug :))\r\n4. Go to ``vagrant/base/analytic_jenkins`` and issue ``VAGRANT_JENKINS_LOCAL_VARS_FILE=your/path vagrant up``\r\n5. Expected result is that you will get a working jenkins istance that you'll be able to log in to ``192.168.33.15:8080`` or ``localhost:8080``. \r\n6. You'll be able to login using ``jenkins:password you set`` credentials to the instance. \r\n7. There will be a role called ``seed`` which role was already ran single time with success result.\r\n8. During the run ``seed`` role created a new role called: ``stub-analytics-jobs``. \r\n9. Remove the key you added to github in step 2. :) \r\n\r\n# Documentation check \r\n\r\n1. Check documentation of the new role ``playbooks/roles/jenkins_analytics/README.md`` to check whether you understand how to configure the this role. \r\n2. Check outstanding issues to check which ones are grievous enough to fix right now :) \r\n\r\n","created_at":"2016-02-25T11:17:54Z","updated_at":"2016-03-18T19:09:14Z","closed_at":null,"merged_at":null,"merge_commit_sha":"c9ce6618a421fb16a4b3dfa67ebfac6b788c5b3a","assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/edx/configuration/pulls/2810/commits","review_comments_url":"https://api.github.com/repos/edx/configuration/pulls/2810/comments","review_comment_url":"https://api.github.com/repos/edx/configuration/pulls/comments{/number}","comments_url":"https://api.github.com/repos/edx/configuration/issues/2810/comments","statuses_url":"https://api.github.com/repos/edx/configuration/statuses/428cd59bae5173a740aff208a68d92d2ed5d7add","head":{"label":"edx:jbzdak/olive-19-auto-provision-analytics-devstack","ref":"jbzdak/olive-19-auto-provision-analytics-devstack","sha":"428cd59bae5173a740aff208a68d92d2ed5d7add","user":{"login":"edx","id":3179841,"avatar_url":"https://avatars.githubusercontent.com/u/3179841?v=3","gravatar_id":"","url":"https://api.github.com/users/edx","html_url":"https://github.com/edx","followers_url":"https://api.github.com/users/edx/followers","following_url":"https://api.github.com/users/edx/following{/other_user}","gists_url":"https://api.github.com/users/edx/gists{/gist_id}","starred_url":"https://api.github.com/users/edx/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/edx/subscriptions","organizations_url":"https://api.github.com/users/edx/orgs","repos_url":"https://api.github.com/users/edx/repos","events_url":"https://api.github.com/users/edx/events{/privacy}","received_events_url":"https://api.github.com/users/edx/received_events","type":"Organization","site_admin":false},"repo":{"id":9478219,"name":"configuration","full_name":"edx/configuration","owner":{"login":"edx","id":3179841,"avatar_url":"https://avatars.githubusercontent.com/u/3179841?v=3","gravatar_id":"","url":"https://api.github.com/users/edx","html_url":"https://github.com/edx","followers_url":"https://api.github.com/users/edx/followers","following_url":"https://api.github.com/users/edx/following{/other_user}","gists_url":"https://api.github.com/users/edx/gists{/gist_id}","starred_url":"https://api.github.com/users/edx/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/edx/subscriptions","organizations_url":"https://api.github.com/users/edx/orgs","repos_url":"https://api.github.com/users/edx/repos","events_url":"https://api.github.com/users/edx/events{/privacy}","received_events_url":"https://api.github.com/users/edx/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/edx/configuration","description":"a simple, but flexible, way for anyone to stand up an instance of the edX platform that is fully configured and ready-to-go","fork":false,"url":"https://api.github.com/repos/edx/configuration","forks_url":"https://api.github.com/repos/edx/configuration/forks","keys_url":"https://api.github.com/repos/edx/configuration/keys{/key_id}","collaborators_url":"https://api.github.com/repos/edx/configuration/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/edx/configuration/teams","hooks_url":"https://api.github.com/repos/edx/configuration/hooks","issue_events_url":"https://api.github.com/repos/edx/configuration/issues/events{/number}","events_url":"https://api.github.com/repos/edx/configuration/events","assignees_url":"https://api.github.com/repos/edx/configuration/assignees{/user}","branches_url":"https://api.github.com/repos/edx/configuration/branches{/branch}","tags_url":"https://api.github.com/repos/edx/configuration/tags","blobs_url":"https://api.github.com/repos/edx/configuration/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/edx/configuration/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/edx/configuration/git/refs{/sha}","trees_url":"https://api.github.com/repos/edx/configuration/git/trees{/sha}","statuses_url":"https://api.github.com/repos/edx/configuration/statuses/{sha}","languages_url":"https://api.github.com/repos/edx/configuration/languages","stargazers_url":"https://api.github.com/repos/edx/configuration/stargazers","contributors_url":"https://api.github.com/repos/edx/configuration/contributors","subscribers_url":"https://api.github.com/repos/edx/configuration/subscribers","subscription_url":"https://api.github.com/repos/edx/configuration/subscription","commits_url":"https://api.github.com/repos/edx/configuration/commits{/sha}","git_commits_url":"https://api.github.com/repos/edx/configuration/git/commits{/sha}","comments_url":"https://api.github.com/repos/edx/configuration/comments{/number}","issue_comment_url":"https://api.github.com/repos/edx/configuration/issues/comments{/number}","contents_url":"https://api.github.com/repos/edx/configuration/contents/{+path}","compare_url":"https://api.github.com/repos/edx/configuration/compare/{base}...{head}","merges_url":"https://api.github.com/repos/edx/configuration/merges","archive_url":"https://api.github.com/repos/edx/configuration/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/edx/configuration/downloads","issues_url":"https://api.github.com/repos/edx/configuration/issues{/number}","pulls_url":"https://api.github.com/repos/edx/configuration/pulls{/number}","milestones_url":"https://api.github.com/repos/edx/configuration/milestones{/number}","notifications_url":"https://api.github.com/repos/edx/configuration/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/edx/configuration/labels{/name}","releases_url":"https://api.github.com/repos/edx/configuration/releases{/id}","deployments_url":"https://api.github.com/repos/edx/configuration/deployments","created_at":"2013-04-16T17:23:42Z","updated_at":"2016-03-13T22:44:49Z","pushed_at":"2016-03-18T15:33:55Z","git_url":"git://github.com/edx/configuration.git","ssh_url":"git@github.com:edx/configuration.git","clone_url":"https://github.com/edx/configuration.git","svn_url":"https://github.com/edx/configuration","homepage":"","size":11306,"stargazers_count":498,"watchers_count":498,"language":"Python","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":466,"mirror_url":null,"open_issues_count":129,"forks":466,"open_issues":129,"watchers":498,"default_branch":"master"}},"base":{"label":"edx:master","ref":"master","sha":"58a95c5567f4127e1c00faefd5becd338cc22730","user":{"login":"edx","id":3179841,"avatar_url":"https://avatars.githubusercontent.com/u/3179841?v=3","gravatar_id":"","url":"https://api.github.com/users/edx","html_url":"https://github.com/edx","followers_url":"https://api.github.com/users/edx/followers","following_url":"https://api.github.com/users/edx/following{/other_user}","gists_url":"https://api.github.com/users/edx/gists{/gist_id}","starred_url":"https://api.github.com/users/edx/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/edx/subscriptions","organizations_url":"https://api.github.com/users/edx/orgs","repos_url":"https://api.github.com/users/edx/repos","events_url":"https://api.github.com/users/edx/events{/privacy}","received_events_url":"https://api.github.com/users/edx/received_events","type":"Organization","site_admin":false},"repo":{"id":9478219,"name":"configuration","full_name":"edx/configuration","owner":{"login":"edx","id":3179841,"avatar_url":"https://avatars.githubusercontent.com/u/3179841?v=3","gravatar_id":"","url":"https://api.github.com/users/edx","html_url":"https://github.com/edx","followers_url":"https://api.github.com/users/edx/followers","following_url":"https://api.github.com/users/edx/following{/other_user}","gists_url":"https://api.github.com/users/edx/gists{/gist_id}","starred_url":"https://api.github.com/users/edx/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/edx/subscriptions","organizations_url":"https://api.github.com/users/edx/orgs","repos_url":"https://api.github.com/users/edx/repos","events_url":"https://api.github.com/users/edx/events{/privacy}","received_events_url":"https://api.github.com/users/edx/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/edx/configuration","description":"a simple, but flexible, way for anyone to stand up an instance of the edX platform that is fully configured and ready-to-go","fork":false,"url":"https://api.github.com/repos/edx/configuration","forks_url":"https://api.github.com/repos/edx/configuration/forks","keys_url":"https://api.github.com/repos/edx/configuration/keys{/key_id}","collaborators_url":"https://api.github.com/repos/edx/configuration/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/edx/configuration/teams","hooks_url":"https://api.github.com/repos/edx/configuration/hooks","issue_events_url":"https://api.github.com/repos/edx/configuration/issues/events{/number}","events_url":"https://api.github.com/repos/edx/configuration/events","assignees_url":"https://api.github.com/repos/edx/configuration/assignees{/user}","branches_url":"https://api.github.com/repos/edx/configuration/branches{/branch}","tags_url":"https://api.github.com/repos/edx/configuration/tags","blobs_url":"https://api.github.com/repos/edx/configuration/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/edx/configuration/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/edx/configuration/git/refs{/sha}","trees_url":"https://api.github.com/repos/edx/configuration/git/trees{/sha}","statuses_url":"https://api.github.com/repos/edx/configuration/statuses/{sha}","languages_url":"https://api.github.com/repos/edx/configuration/languages","stargazers_url":"https://api.github.com/repos/edx/configuration/stargazers","contributors_url":"https://api.github.com/repos/edx/configuration/contributors","subscribers_url":"https://api.github.com/repos/edx/configuration/subscribers","subscription_url":"https://api.github.com/repos/edx/configuration/subscription","commits_url":"https://api.github.com/repos/edx/configuration/commits{/sha}","git_commits_url":"https://api.github.com/repos/edx/configuration/git/commits{/sha}","comments_url":"https://api.github.com/repos/edx/configuration/comments{/number}","issue_comment_url":"https://api.github.com/repos/edx/configuration/issues/comments{/number}","contents_url":"https://api.github.com/repos/edx/configuration/contents/{+path}","compare_url":"https://api.github.com/repos/edx/configuration/compare/{base}...{head}","merges_url":"https://api.github.com/repos/edx/configuration/merges","archive_url":"https://api.github.com/repos/edx/configuration/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/edx/configuration/downloads","issues_url":"https://api.github.com/repos/edx/configuration/issues{/number}","pulls_url":"https://api.github.com/repos/edx/configuration/pulls{/number}","milestones_url":"https://api.github.com/repos/edx/configuration/milestones{/number}","notifications_url":"https://api.github.com/repos/edx/configuration/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/edx/configuration/labels{/name}","releases_url":"https://api.github.com/repos/edx/configuration/releases{/id}","deployments_url":"https://api.github.com/repos/edx/configuration/deployments","created_at":"2013-04-16T17:23:42Z","updated_at":"2016-03-13T22:44:49Z","pushed_at":"2016-03-18T15:33:55Z","git_url":"git://github.com/edx/configuration.git","ssh_url":"git@github.com:edx/configuration.git","clone_url":"https://github.com/edx/configuration.git","svn_url":"https://github.com/edx/configuration","homepage":"","size":11306,"stargazers_count":498,"watchers_count":498,"language":"Python","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":466,"mirror_url":null,"open_issues_count":129,"forks":466,"open_issues":129,"watchers":498,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/edx/configuration/pulls/2810"},"html":{"href":"https://github.com/edx/configuration/pull/2810"},"issue":{"href":"https://api.github.com/repos/edx/configuration/issues/2810"},"comments":{"href":"https://api.github.com/repos/edx/configuration/issues/2810/comments"},"review_comments":{"href":"https://api.github.com/repos/edx/configuration/pulls/2810/comments"},"review_comment":{"href":"https://api.github.com/repos/edx/configuration/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/edx/configuration/pulls/2810/commits"},"statuses":{"href":"https://api.github.com/repos/edx/configuration/statuses/428cd59bae5173a740aff208a68d92d2ed5d7add"}}}}