{"action":"created","issue":{"url":"https://api.github.com/repos/edgexfoundry/edgex-go/issues/2146","repository_url":"https://api.github.com/repos/edgexfoundry/edgex-go","labels_url":"https://api.github.com/repos/edgexfoundry/edgex-go/issues/2146/labels{/name}","comments_url":"https://api.github.com/repos/edgexfoundry/edgex-go/issues/2146/comments","events_url":"https://api.github.com/repos/edgexfoundry/edgex-go/issues/2146/events","html_url":"https://github.com/edgexfoundry/edgex-go/pull/2146","id":524688190,"node_id":"MDExOlB1bGxSZXF1ZXN0MzQyNDE4NDkw","number":2146,"title":"Split customized vault into two docker images- part 1","user":{"login":"jim-wang-intel","id":25697718,"node_id":"MDQ6VXNlcjI1Njk3NzE4","avatar_url":"https://avatars2.githubusercontent.com/u/25697718?v=4","gravatar_id":"","url":"https://api.github.com/users/jim-wang-intel","html_url":"https://github.com/jim-wang-intel","followers_url":"https://api.github.com/users/jim-wang-intel/followers","following_url":"https://api.github.com/users/jim-wang-intel/following{/other_user}","gists_url":"https://api.github.com/users/jim-wang-intel/gists{/gist_id}","starred_url":"https://api.github.com/users/jim-wang-intel/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jim-wang-intel/subscriptions","organizations_url":"https://api.github.com/users/jim-wang-intel/orgs","repos_url":"https://api.github.com/users/jim-wang-intel/repos","events_url":"https://api.github.com/users/jim-wang-intel/events{/privacy}","received_events_url":"https://api.github.com/users/jim-wang-intel/received_events","type":"User","site_admin":false},"labels":[{"id":1463695951,"node_id":"MDU6TGFiZWwxNDYzNjk1OTUx","url":"https://api.github.com/repos/edgexfoundry/edgex-go/labels/geneva","name":"geneva","color":"0e8a16","default":false,"description":""},{"id":1433045037,"node_id":"MDU6TGFiZWwxNDMzMDQ1MDM3","url":"https://api.github.com/repos/edgexfoundry/edgex-go/labels/security-services","name":"security-services","color":"bfdadc","default":false,"description":""}],"state":"open","locked":false,"assignee":{"login":"jim-wang-intel","id":25697718,"node_id":"MDQ6VXNlcjI1Njk3NzE4","avatar_url":"https://avatars2.githubusercontent.com/u/25697718?v=4","gravatar_id":"","url":"https://api.github.com/users/jim-wang-intel","html_url":"https://github.com/jim-wang-intel","followers_url":"https://api.github.com/users/jim-wang-intel/followers","following_url":"https://api.github.com/users/jim-wang-intel/following{/other_user}","gists_url":"https://api.github.com/users/jim-wang-intel/gists{/gist_id}","starred_url":"https://api.github.com/users/jim-wang-intel/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jim-wang-intel/subscriptions","organizations_url":"https://api.github.com/users/jim-wang-intel/orgs","repos_url":"https://api.github.com/users/jim-wang-intel/repos","events_url":"https://api.github.com/users/jim-wang-intel/events{/privacy}","received_events_url":"https://api.github.com/users/jim-wang-intel/received_events","type":"User","site_admin":false},"assignees":[{"login":"jim-wang-intel","id":25697718,"node_id":"MDQ6VXNlcjI1Njk3NzE4","avatar_url":"https://avatars2.githubusercontent.com/u/25697718?v=4","gravatar_id":"","url":"https://api.github.com/users/jim-wang-intel","html_url":"https://github.com/jim-wang-intel","followers_url":"https://api.github.com/users/jim-wang-intel/followers","following_url":"https://api.github.com/users/jim-wang-intel/following{/other_user}","gists_url":"https://api.github.com/users/jim-wang-intel/gists{/gist_id}","starred_url":"https://api.github.com/users/jim-wang-intel/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jim-wang-intel/subscriptions","organizations_url":"https://api.github.com/users/jim-wang-intel/orgs","repos_url":"https://api.github.com/users/jim-wang-intel/repos","events_url":"https://api.github.com/users/jim-wang-intel/events{/privacy}","received_events_url":"https://api.github.com/users/jim-wang-intel/received_events","type":"User","site_admin":false}],"milestone":{"url":"https://api.github.com/repos/edgexfoundry/edgex-go/milestones/5","html_url":"https://github.com/edgexfoundry/edgex-go/milestone/5","labels_url":"https://api.github.com/repos/edgexfoundry/edgex-go/milestones/5/labels","id":4785024,"node_id":"MDk6TWlsZXN0b25lNDc4NTAyNA==","number":5,"title":"EdgeX Geneva","description":"The Geneva release is likely to focus on 3rd party service certification, platform and setup guidance relative to EdgeX performance measures, and high availability concerns.\r\n\r\nFor more information, please see https://wiki.edgexfoundry.org/display/FA/Geneva+Release","creator":{"login":"brandonforster","id":1308221,"node_id":"MDQ6VXNlcjEzMDgyMjE=","avatar_url":"https://avatars3.githubusercontent.com/u/1308221?v=4","gravatar_id":"","url":"https://api.github.com/users/brandonforster","html_url":"https://github.com/brandonforster","followers_url":"https://api.github.com/users/brandonforster/followers","following_url":"https://api.github.com/users/brandonforster/following{/other_user}","gists_url":"https://api.github.com/users/brandonforster/gists{/gist_id}","starred_url":"https://api.github.com/users/brandonforster/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/brandonforster/subscriptions","organizations_url":"https://api.github.com/users/brandonforster/orgs","repos_url":"https://api.github.com/users/brandonforster/repos","events_url":"https://api.github.com/users/brandonforster/events{/privacy}","received_events_url":"https://api.github.com/users/brandonforster/received_events","type":"User","site_admin":false},"open_issues":136,"closed_issues":26,"state":"open","created_at":"2019-10-25T19:43:03Z","updated_at":"2019-11-19T20:45:54Z","due_on":"2020-04-15T07:00:00Z","closed_at":null},"comments":6,"created_at":"2019-11-18T23:51:01Z","updated_at":"2019-11-19T20:52:22Z","closed_at":null,"author_association":"CONTRIBUTOR","pull_request":{"url":"https://api.github.com/repos/edgexfoundry/edgex-go/pulls/2146","html_url":"https://github.com/edgexfoundry/edgex-go/pull/2146","diff_url":"https://github.com/edgexfoundry/edgex-go/pull/2146.diff","patch_url":"https://github.com/edgexfoundry/edgex-go/pull/2146.patch"},"body":"This PR attempts to address issues #1716.  There are a couple steps to accomplish that:\r\n\r\n1. Split the docker image file from inhering the Vault\r\n2. Update the newly generated TLS paths (Related PRs: https://github.com/edgexfoundry/docker-edgex-mongo/pull/71, )\r\n3. Update the consul checking scripts (Related PR: https://github.com/edgexfoundry/docker-edgex-consul/pull/10)\r\n\r\nThe focus of this PR is on the **step 1**.\r\n\r\n  The original customized edgex-vault docker image is broken into two separate images:\r\n    the official Vault docker image from HashiCorp and the `security-secrets-setup` docker image\r\n\r\n - add customized docker entrypoint script for starting vault\r\n - use sentinel files for readiness check of TLS cert of vault because vault image does not have curl command available\r\n - prepare cross-mount volume /vault/init for customized startup script\r\n - remove creation of /vault/config/assets folder from security-secrets-setup and\r\n   create that folder from entrypoint of security-secretstore-setup (aka vault-worker)\r\n\r\n## Testing steps\r\n1. Build the new docker images for both `security-secrets-setup` and `security-secretstore-setup`:\r\n```sh\r\n$ make docker_security_secrets_setup\r\n$ make docker_security_secretstore_setup\r\n```\r\n2. Run the secrets-setup docker container:\r\n```sh\r\n$ docker run --tmpfs /run  -v /run/edgex/secrets:/run/edgex/secrets -v vault-init:/vault/init edgexfoundry/docker-edgex-secrets-setup-go:1.1.1-dev\r\n```\r\nIf running ok, one should expect to see console outputs like this:\r\n```sh\r\nXDG_RUNTIME_DIR /run/user/0\r\nBASE_DIR /edgex/security-secrets-setup\r\nExecuting security-secrets-setup generate\r\nlevel=ERROR ts=2019-11-18T23:40:17.197784157Z app=edgex-security-secrets-setup source=logger.go:73 msg=\"logTarget cannot be blank, using stdout only\"\r\nCreating directory: /edgex/security-secrets-setup/logs\r\n{\r\n    \"CACertFile\": \"/run/user/0/edgex/security-secrets-setup/scratch/config/pki/EdgeXFoundryCA/EdgeXFoundryCA.pem\",\r\n    \"CACountry\": \"US\",\r\n    \"CAKeyFile\": \"/run/user/0/edgex/security-secrets-setup/scratch/config/pki/EdgeXFoundryCA/EdgeXFoundryCA.priv.key\",\r\n    \"CALocality\": \"San Francisco\",\r\n    \"CAName\": \"EdgeXFoundryCA\",\r\n    \"CAOrg\": \"EdgeXFoundry\",\r\n    \"CAState\": \"CA\",\r\n    \"DumpKeys\": false,\r\n    \"ECCurve\": 384,\r\n    \"ECScheme\": true,\r\n    \"NewCA\": true,\r\n    \"RSAKeySize\": 4096,\r\n    \"RSAScheme\": false,\r\n    \"TLSAltFqdn\": \"edgex-vault.local\",\r\n    \"TLSCertFile\": \"/run/user/0/edgex/security-secrets-setup/scratch/config/pki/EdgeXFoundryCA/edgex-vault.pem\",\r\n    \"TLSCountry\": \"US\",\r\n    \"TLSDomain\": \"local\",\r\n    \"TLSFqdn\": \"edgex-vault\",\r\n    \"TLSHost\": \"edgex-vault\",\r\n    \"TLSKeyFile\": \"/run/user/0/edgex/security-secrets-setup/scratch/config/pki/EdgeXFoundryCA/edgex-vault.priv.key\",\r\n    \"TLSLocality\": \"San Francisco\",\r\n    \"TLSOrg\": \"Vault\",\r\n    \"TLSState\": \"CA\"\r\n}\r\n{\r\n    \"CACertFile\": \"/run/user/0/edgex/security-secrets-setup/scratch/config/pki/EdgeXFoundryCA/EdgeXFoundryCA.pem\",\r\n    \"CACountry\": \"US\",\r\n    \"CAKeyFile\": \"/run/user/0/edgex/security-secrets-setup/scratch/config/pki/EdgeXFoundryCA/EdgeXFoundryCA.priv.key\",\r\n    \"CALocality\": \"San Francisco\",\r\n    \"CAName\": \"EdgeXFoundryCA\",\r\n    \"CAOrg\": \"EdgeXFoundry\",\r\n    \"CAState\": \"CA\",\r\n    \"DumpKeys\": false,\r\n    \"ECCurve\": 384,\r\n    \"ECScheme\": true,\r\n    \"NewCA\": false,\r\n    \"RSAKeySize\": 4096,\r\n    \"RSAScheme\": false,\r\n    \"TLSAltFqdn\": \"edgex-kong.local\",\r\n    \"TLSCertFile\": \"/run/user/0/edgex/security-secrets-setup/scratch/config/pki/EdgeXFoundryCA/edgex-kong.pem\",\r\n    \"TLSCountry\": \"US\",\r\n    \"TLSDomain\": \"local\",\r\n    \"TLSFqdn\": \"edgex-kong\",\r\n    \"TLSHost\": \"edgex-kong\",\r\n    \"TLSKeyFile\": \"/run/user/0/edgex/security-secrets-setup/scratch/config/pki/EdgeXFoundryCA/edgex-kong.priv.key\",\r\n    \"TLSLocality\": \"San Francisco\",\r\n    \"TLSOrg\": \"Kong\",\r\n    \"TLSState\": \"CA\"\r\n}\r\nlevel=INFO ts=2019-11-18T23:40:17.241470347Z app=edgex-security-secrets-setup source=generate.go:211 msg=\"pki-init generation completes\"\r\nInstalling Vault's startup script\r\nExecuting hook=chown -R 100:1000 /run/edgex/secrets/edgex-vault/ /vault/init/\r\n```\r\n\r\n3. Run consul: you will need one from Fuji branch and modify the security-secrets-setup-checker.sh to check the sentinel files like:\r\n```sh\r\n#!/bin/sh -ex\r\n\r\n# check that the root CA exists, the vault and kong certs exist\r\n# the new way of checking just the sentinel files which will have all TLS assets\r\ntest -f /run/edgex/secrets/ca/.security-secrets-setup.complete\r\ntest -f /run/edgex/secrets/edgex-vault/.security-secrets-setup.complete\r\ntest -f /run/edgex/secrets/edgex-kong/.security-secrets-setup.complete\r\n```\r\nand then do a docker build `make docker`.\r\nThen run edgex-consul docker container like\r\n```sh\r\ndocker run --rm --network edgex-network --net-alias edgex-core-consul -v consul-config:/consul/config -v consul-data:/consul/data -v consul-scripts:/consul/scripts -v vault-config:/vault/config -v /run/edgex/secrets:/run/edgex/secrets:ro -p 8400:8400 -p 8500:8500 edgexfoundry/docker-edgex-consul:1.1.0\r\n```\r\nmake sure edgex-consul runs up ok.\r\n\r\n4. Finally, starts up the Docker Hashicorp Vault like:\r\n```sh\r\ndocker run --rm --privileged --entrypoint \"/vault/init/start_vault.sh\" --network edgex-network --net-alias edgex-vault -p 8200:8200 -e VAULT_ADDR=https://edgex-vault:8200 -e VAULT_CONFIG_DIR=/vault/config -e VAULT_UI=true -v vault-file:/vault/file -v vault-logs:/vault/logs -v vault-init:/vault/init:ro -v /run/edgex/secrets/edgex-vault:/run/edgex/secrets/edgex-vault:ro vault:1.0.3\r\n```\r\n\r\nIf everything works, one should expect to see Vault starts up listening like:\r\n```sh\r\nVAULT_LOCAL_CONFIG: listener \"tcp\" { address = \"edgex-vault:8200\" tls_disable = \"0\" cluster_address = \"edgex-vault:8201\" tls_min_version = \"tls12\" tls_client_ca_file =\"/run/edgex/secrets/edgex-vault/ca.pem\" tls_cert_file =\"/run/edgex/secrets/edgex-vault/server.crt\" tls_key_file = \"/run/edgex/secrets/edgex-vault/server.key\" tls_perfer_server_cipher_suites = \"true\" } backend \"consul\" { path = \"vault/\" address = \"edgex-core-consul:8500\" scheme = \"http\" redirect_addr = \"https://edgex-vault:8200\" cluster_addr = \"https://edgex-vault:8201\" } default_lease_ttl = \"168h\" max_lease_ttl = \"720h\"\r\nStarting edgex-vault...\r\n==> Vault server configuration:\r\n\r\n             Api Address: https://edgex-vault:8200\r\n                     Cgo: disabled\r\n         Cluster Address: https://edgex-vault:8201\r\n              Listener 1: tcp (addr: \"edgex-vault:8200\", cluster address: \"edgex-vault:8201\", max_request_duration: \"1m30s\", max_request_size: \"33554432\", tls: \"enabled\")\r\n               Log Level: info\r\n                   Mlock: supported: true, enabled: true\r\n                 Storage: consul (HA available)\r\n                 Version: Vault v1.0.3\r\n             Version Sha: 85909e3373aa743c34a6a0ab59131f61fd9e8e43\r\n\r\n==> Vault server started! Log data will stream in below:\r\n\r\n```\r\n\r\nSigned-off-by: Jim Wang <yutsung.jim.wang@intel.com>"},"comment":{"url":"https://api.github.com/repos/edgexfoundry/edgex-go/issues/comments/555708054","html_url":"https://github.com/edgexfoundry/edgex-go/pull/2146#issuecomment-555708054","issue_url":"https://api.github.com/repos/edgexfoundry/edgex-go/issues/2146","id":555708054,"node_id":"MDEyOklzc3VlQ29tbWVudDU1NTcwODA1NA==","user":{"login":"bnevis-i","id":21301128,"node_id":"MDQ6VXNlcjIxMzAxMTI4","avatar_url":"https://avatars0.githubusercontent.com/u/21301128?v=4","gravatar_id":"","url":"https://api.github.com/users/bnevis-i","html_url":"https://github.com/bnevis-i","followers_url":"https://api.github.com/users/bnevis-i/followers","following_url":"https://api.github.com/users/bnevis-i/following{/other_user}","gists_url":"https://api.github.com/users/bnevis-i/gists{/gist_id}","starred_url":"https://api.github.com/users/bnevis-i/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bnevis-i/subscriptions","organizations_url":"https://api.github.com/users/bnevis-i/orgs","repos_url":"https://api.github.com/users/bnevis-i/repos","events_url":"https://api.github.com/users/bnevis-i/events{/privacy}","received_events_url":"https://api.github.com/users/bnevis-i/received_events","type":"User","site_admin":false},"created_at":"2019-11-19T20:52:22Z","updated_at":"2019-11-19T20:52:22Z","author_association":"CONTRIBUTOR","body":"> I think we may be able to get it with only mounting the individual files instead of the whole directory. I still like to check the whole files. Maybe some thing to improve in future for the final top level .complete file\r\n\r\nThough it would work, I tend to prefer directory-level secret granularity as it is easier to port to other orchestrators like k8s that only support directory mounts for secrets."}}
{"actor":{"display_login":"bnevis-i"}}