{"action":"created","issue":{"url":"https://api.github.com/repos/dcos/dcos/issues/958","repository_url":"https://api.github.com/repos/dcos/dcos","labels_url":"https://api.github.com/repos/dcos/dcos/issues/958/labels{/name}","comments_url":"https://api.github.com/repos/dcos/dcos/issues/958/comments","events_url":"https://api.github.com/repos/dcos/dcos/issues/958/events","html_url":"https://github.com/dcos/dcos/pull/958","id":189523042,"number":958,"title":"gen: Add the concept of a Resolvable, Resolve config per-Target","user":{"login":"cmaloney","id":751088,"avatar_url":"https://avatars.githubusercontent.com/u/751088?v=3","gravatar_id":"","url":"https://api.github.com/users/cmaloney","html_url":"https://github.com/cmaloney","followers_url":"https://api.github.com/users/cmaloney/followers","following_url":"https://api.github.com/users/cmaloney/following{/other_user}","gists_url":"https://api.github.com/users/cmaloney/gists{/gist_id}","starred_url":"https://api.github.com/users/cmaloney/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cmaloney/subscriptions","organizations_url":"https://api.github.com/users/cmaloney/orgs","repos_url":"https://api.github.com/users/cmaloney/repos","events_url":"https://api.github.com/users/cmaloney/events{/privacy}","received_events_url":"https://api.github.com/users/cmaloney/received_events","type":"User","site_admin":false},"labels":[{"id":364878184,"url":"https://api.github.com/repos/dcos/dcos/labels/Ready%20For%20Review","name":"Ready For Review","color":"0e8a16","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2016-11-15T22:17:12Z","updated_at":"2016-11-22T06:36:54Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/dcos/dcos/pulls/958","html_url":"https://github.com/dcos/dcos/pull/958","diff_url":"https://github.com/dcos/dcos/pull/958.diff","patch_url":"https://github.com/dcos/dcos/pull/958.patch"},"body":"\r\n - Targets now get finalized with results. This also means they need to be constructued per-use.\r\n - validate_configuration() no longer exists, as resolve_configuration does exactly the same thing,\r\n   users of validate_configuration() are migrated to just use resolve_configuration() once.\r\n - The per-target results aren't yet plugged into various places, that will come in a later pr\r\n - Introduce a notion of variables \"visited\" to calculate a particular variable.\r\n - Introduce a concept of a \"Resolvable\". A resolvable tracks the resolution state of a particular configuration value. (Unresolved, being resolved, error). It will eventually also include a \"late resolution\" (late binding) state. The introduction centralizes the logic for tracking that, and makes it harder to get wrong. Resolvables also keep track of what caused them to come into existence / get resolved.\r\n - Switches / sub-scopes now generate single-argument \"validate\" functions rather than having custom logic to validate their values when we reached the switch they were used in (See yield_implicit_validates)\r\n - Renames DFSArgumentCalculator to Resolver. A resolver is now only usable once, takes a set of setters (taken from a set of sources) and targets, then resolves them all at once. The goal being that the resolution of all targets produces a single, consistent / valid against eachother full final configuration. Resolver now owns / builds the status dictionary itself, and makes it accessible to callers\r\n - Adds a new unit test for gen internals code\r\n\r\nThings this works towards enabling\r\n - Typed parameters. \"Resolvables\" can be focused around a type (as discovered by python type annotations on function parameters)\r\n - \"nested\" configuration. Currently all config must be one top level dictionary. In theory though we could use nested dictionaries and flatten them ({\"a\": {\"b\": {\"c\": \"d\"}}} could be accessed as `a_b_c`.\r\n - Late-binding. Resolvables do a much better job of tracking / keeping contained the resolution state, code which was currently spread about DFSArgumentCalculator previously. The code to track \"unset\" and spread it through the tree, is very similar to the code needed to track \"late bound\" and that property spreading / chaining through the tree.\r\n - \"secret\" / private data such as aws keys in config.yaml. Things like the AWS keys should be explicitly protected / marked so they don't show up in things like the user config.yaml inside the cluster / on every host. If anything would make them show up, that should result in an error.\r\n\r\n# Issues\r\n\r\n - Late-binding configuration\r\n - Secrets in dcos-release.config.yaml vs. config.yaml for AWS Advanced templates\r\n - Enabling dcos-launch to use config.yaml\r\n\r\n# Checklist\r\n\r\n - [x] Included a test which will fail if code is reverted but test is not\r\n - [x] Followed relevant code rules [Rules for Packages and Systemd](https://github.com/dcos/dcos/tree/master/docs)\r\n"},"comment":{"url":"https://api.github.com/repos/dcos/dcos/issues/comments/262159825","html_url":"https://github.com/dcos/dcos/pull/958#issuecomment-262159825","issue_url":"https://api.github.com/repos/dcos/dcos/issues/958","id":262159825,"user":{"login":"cmaloney","id":751088,"avatar_url":"https://avatars.githubusercontent.com/u/751088?v=3","gravatar_id":"","url":"https://api.github.com/users/cmaloney","html_url":"https://github.com/cmaloney","followers_url":"https://api.github.com/users/cmaloney/followers","following_url":"https://api.github.com/users/cmaloney/following{/other_user}","gists_url":"https://api.github.com/users/cmaloney/gists{/gist_id}","starred_url":"https://api.github.com/users/cmaloney/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cmaloney/subscriptions","organizations_url":"https://api.github.com/users/cmaloney/orgs","repos_url":"https://api.github.com/users/cmaloney/repos","events_url":"https://api.github.com/users/cmaloney/events{/privacy}","received_events_url":"https://api.github.com/users/cmaloney/received_events","type":"User","site_admin":false},"created_at":"2016-11-22T06:36:54Z","updated_at":"2016-11-22T06:36:54Z","body":"@branden pulled validation out into a class in a second commit, adds a bit of lines of code and the API is a bit wonky (The multi-arg validate results have to be yielded out while the single-arg ones don't). Not sure if it's worth doing at this point."}}
{"actor":{"display_login":"cmaloney"}}