{"action":"created","issue":{"url":"https://api.github.com/repos/manifoldco/heighliner/issues/91","repository_url":"https://api.github.com/repos/manifoldco/heighliner","labels_url":"https://api.github.com/repos/manifoldco/heighliner/issues/91/labels{/name}","comments_url":"https://api.github.com/repos/manifoldco/heighliner/issues/91/comments","events_url":"https://api.github.com/repos/manifoldco/heighliner/issues/91/events","html_url":"https://github.com/manifoldco/heighliner/issues/91","id":341341126,"node_id":"MDU6SXNzdWUzNDEzNDExMjY=","number":91,"title":"Implement proper UpdateStrategy for Microservices","user":{"login":"jelmersnoeck","id":815655,"node_id":"MDQ6VXNlcjgxNTY1NQ==","avatar_url":"https://avatars1.githubusercontent.com/u/815655?v=4","gravatar_id":"","url":"https://api.github.com/users/jelmersnoeck","html_url":"https://github.com/jelmersnoeck","followers_url":"https://api.github.com/users/jelmersnoeck/followers","following_url":"https://api.github.com/users/jelmersnoeck/following{/other_user}","gists_url":"https://api.github.com/users/jelmersnoeck/gists{/gist_id}","starred_url":"https://api.github.com/users/jelmersnoeck/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jelmersnoeck/subscriptions","organizations_url":"https://api.github.com/users/jelmersnoeck/orgs","repos_url":"https://api.github.com/users/jelmersnoeck/repos","events_url":"https://api.github.com/users/jelmersnoeck/events{/privacy}","received_events_url":"https://api.github.com/users/jelmersnoeck/received_events","type":"User","site_admin":false},"labels":[{"id":848787518,"node_id":"MDU6TGFiZWw4NDg3ODc1MTg=","url":"https://api.github.com/repos/manifoldco/heighliner/labels/bug","name":"bug","color":"d73a4a","default":true},{"id":848787520,"node_id":"MDU6TGFiZWw4NDg3ODc1MjA=","url":"https://api.github.com/repos/manifoldco/heighliner/labels/enhancement","name":"enhancement","color":"a2eeef","default":true}],"state":"open","locked":false,"assignee":{"login":"jelmersnoeck","id":815655,"node_id":"MDQ6VXNlcjgxNTY1NQ==","avatar_url":"https://avatars1.githubusercontent.com/u/815655?v=4","gravatar_id":"","url":"https://api.github.com/users/jelmersnoeck","html_url":"https://github.com/jelmersnoeck","followers_url":"https://api.github.com/users/jelmersnoeck/followers","following_url":"https://api.github.com/users/jelmersnoeck/following{/other_user}","gists_url":"https://api.github.com/users/jelmersnoeck/gists{/gist_id}","starred_url":"https://api.github.com/users/jelmersnoeck/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jelmersnoeck/subscriptions","organizations_url":"https://api.github.com/users/jelmersnoeck/orgs","repos_url":"https://api.github.com/users/jelmersnoeck/repos","events_url":"https://api.github.com/users/jelmersnoeck/events{/privacy}","received_events_url":"https://api.github.com/users/jelmersnoeck/received_events","type":"User","site_admin":false},"assignees":[{"login":"jelmersnoeck","id":815655,"node_id":"MDQ6VXNlcjgxNTY1NQ==","avatar_url":"https://avatars1.githubusercontent.com/u/815655?v=4","gravatar_id":"","url":"https://api.github.com/users/jelmersnoeck","html_url":"https://github.com/jelmersnoeck","followers_url":"https://api.github.com/users/jelmersnoeck/followers","following_url":"https://api.github.com/users/jelmersnoeck/following{/other_user}","gists_url":"https://api.github.com/users/jelmersnoeck/gists{/gist_id}","starred_url":"https://api.github.com/users/jelmersnoeck/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jelmersnoeck/subscriptions","organizations_url":"https://api.github.com/users/jelmersnoeck/orgs","repos_url":"https://api.github.com/users/jelmersnoeck/repos","events_url":"https://api.github.com/users/jelmersnoeck/events{/privacy}","received_events_url":"https://api.github.com/users/jelmersnoeck/received_events","type":"User","site_admin":false}],"milestone":null,"comments":0,"created_at":"2018-07-15T18:26:45Z","updated_at":"2018-07-16T14:12:44Z","closed_at":null,"author_association":"MEMBER","body":"# Rolling Updates\r\n\r\nCurrently, we're not doing rolling updates nicely. We cut off as soon as a new\r\nrelease is available in the Microservice, but that doesn't mean it's actually\r\navailable yet (boot times etc.)\r\n\r\nWe should take several steps to make this a proper rolling update.\r\n\r\n## UpdateStrategy\r\n\r\nWe'll want to move the `UpdateStrategy` from the `NetworkPolicy` to the\r\n`Microservice`. The main motivator for this is that we will eventually also have\r\nnon HTTP applications running through Heighliner like background workers and\r\ntemporary jobs.\r\n\r\nFrom an availability point of view, the `Microservice` is the best point to\r\ndetermine which release should actually be marked as the `active` release. In an\r\nHTTP scenarion, this means that this is the release to which the traffic should\r\nlead to.\r\n\r\n### Minimum Available\r\n\r\nThe UpdateStrategy should have a `MinimumAvailable` field, allowing the user to\r\nspecify how many versions they want to keep alive. This includes the active\r\nversion.\r\n\r\nFor rolling upgrades, if the version is set to 1, we will not delete the older\r\nactive version until the new version is in a state where we're happy with\r\navailability.\r\n\r\nThe default value will be 3.\r\n\r\n### Strategy: latest\r\n\r\nThe `latest` strategy is fairly simple and doesn't currently take any arguments.\r\nIt will look at available images for the `ReleaseGroup` and deploy a new version\r\nif one is available in the `ImagePolicy`. Once this new version is deployed and\r\navailable to serve requests, it will automatically be marked as `active` (see\r\nmore on statuses [here](#Release)). Other versions will be cycled accordingly.\r\n\r\n### Strategy: manual\r\n\r\n`manual` takes priority over `latest`. Both can be set, but when manual is set\r\nthe cluster will choose this as the desired version to be marked as active\r\n(see more on statuses [here](#Release)).\r\n\r\n## Microservice Changes\r\n\r\nTo reflect the UpdateStrategy, I propose we add the following to the\r\n`Microservice.Spec`:\r\n\r\n```go\r\n// MicroserviceSpec represents the specification for a Microservice. It houses\r\n// all the policies which we'll use to build a VersionedMicroservice.\r\ntype MicroserviceSpec struct {\r\n\t// …\r\n\r\n\t// UpdateStrategy represents the way this Microservice will roll over from\r\n\t// one version to another.\r\n\tUpdateStrategy UpdateStrategy `json:\"updateStrategy\"`\r\n}\r\n```\r\n\r\nWe will keep using the existing `UpdateStrategy`:\r\n\r\n```go\r\ntype UpdateStrategy struct {\r\n\tManual *ManualUpdateStrategy `json:\"manual\"`\r\n\tLatest *LatestUpdateStrategy `json:\"latest\"`\r\n}\r\n\r\ntype ManualUpdateStrategy struct {\r\n\tSemVer *SemVerRelease `json:\"semVer\"`\r\n}\r\n\r\ntype LatestUpdateStrategy struct{}\r\n```\r\n\r\n\r\nWe also want to update the `Microservice.Status` data format. We want to\r\nintroduce the concept of `ReleaseGroup`. We internally already do this at the\r\n`NetworkPolicy` level, but we should do this a level higher up so it's easily\r\nvisible to the rest of the cluster.\r\n\r\n```go\r\n// MicroserviceStatus represents the status a specific Microservice is in.\r\ntype MicroserviceStatus struct {\r\n\tReleaseGroups []ReleaseGroup `json:\"releaseGroups\"`\r\n}\r\n```\r\n\r\n### ReleaseGroup\r\n\r\nA `ReleaseGroup` is a grouping of releases that fall within a specific release\r\nlevel. For example, when a cluster is configured to monitor `Preview` releases,\r\na specific release group would be a single PR. We can have multiple PRs, each\r\nPR would be it's own `ReleaseGroup`.\r\n\r\nWhen we choose to use a `ReleaseMinor` versioning level, a release group could\r\nbe `1.2.x` or it could be `1.3.x`. All the versions within the `x` range would\r\nthen fall within that specific group.\r\n\r\n```go\r\n// ReleaseGroup represents a group of releases that are tied together through\r\n// their versioning configuration. A ReleaseGroup could be represented by a\r\n// single Pull Request like `add-feature-x`, or it could be represented by a\r\n// specific tag version, like `1.2.x`.\r\ntype ReleaseGroup struct {\r\n\t// Name represents the name fo the release group. For PRs this will be\r\n\t// represented by the Pull Request name. For (Pre)Releases this will be\r\n\t// represented by the filtered tag.\r\n\tName string `json:\"name\"`\r\n\r\n\t// Releases represents the releases that are available for the specific\r\n\t// ReleaseGroup.\r\n\tReleases []Release `json:\"release\"`\r\n}\r\n```\r\n\r\n### <a name=\"release\"></a> Release\r\n\r\nThe Release itself will reflect a minor change as well. We'll add a `Status`\r\nfield to the release. We can have several Statuses:\r\n\r\n- `Active`: the main active release, the release we actually want to use\r\n- `Canary`: a temporary deployment which is active, but doesn't receive the main trafic\r\n- `Error`: a temporary deployment which isn't active for some reason\r\n- `Deprecated`: a release that will be removed shortly\r\n\r\n```go\r\ntype ReleaseStatus string\r\n\r\nconst (\r\n\t// ReleaseStatusActive refers to a release which receives the application's\r\n\t// main traffic.\r\n\tReleaseStatusActive ReleaseStatus = \"active\"\r\n\r\n\t// ReleaseStatusCanary refers to a release which is healthy and receives the\r\n\t// application's test traffic.\r\n\tReleaseStatusCanary ReleaseStatus = \"canary\"\r\n\r\n\t// ReleaseStatusError refers to a release which encountered an error whilst\r\n\t// trying to deploy.\r\n\tReleaseStatusError ReleaseStatus = \"error\"\r\n\r\n\t// ReleaseStatusDeprecated referes to a release which will be removed\r\n\t// shortly.\r\n\tReleaseStatusDreprecated ReleaseStatus = \"deprecated\"\r\n)\r\n```\r\n\r\n```go\r\n// Release represents a specific release for a version of an image.\r\ntype Release struct {\r\n\t// …\r\n\r\n\t// Status represents the current status of this specific Release.\r\n\tStatus ReleaseStatus `json:\"status\"`\r\n}\r\n```\r\n\r\n## NetworkPolicy Changes\r\n\r\nIn the `NetworkPolicy.Spec`, we'd remove the `UpdateStrategy` field and always\r\nrely on the `active` release for a `ReleaseGroup` in the `Microservice.Status`.\r\n\r\nWe'll do what we do now and use the labels of `Microservice Release` where the\r\nStatus is set to `Active`. We'll then update the main Service to point to these\r\nlabels instead.\r\n\r\nBy doing this, we should achieve 0 downtime as the `Microservice` controller\r\nonly marks the Release Active once it's happy with it's availability.\r\n\r\nThe NetworkPolicy will now be simpler and just iterate over the ReleaseGroups.\r\nWe can now introduce a new Template Variable to the templating system for the\r\nDomains as well, `ReleaseGroupName`."},"comment":{"url":"https://api.github.com/repos/manifoldco/heighliner/issues/comments/405261074","html_url":"https://github.com/manifoldco/heighliner/issues/91#issuecomment-405261074","issue_url":"https://api.github.com/repos/manifoldco/heighliner/issues/91","id":405261074,"node_id":"MDEyOklzc3VlQ29tbWVudDQwNTI2MTA3NA==","user":{"login":"jbowes","id":12997,"node_id":"MDQ6VXNlcjEyOTk3","avatar_url":"https://avatars3.githubusercontent.com/u/12997?v=4","gravatar_id":"","url":"https://api.github.com/users/jbowes","html_url":"https://github.com/jbowes","followers_url":"https://api.github.com/users/jbowes/followers","following_url":"https://api.github.com/users/jbowes/following{/other_user}","gists_url":"https://api.github.com/users/jbowes/gists{/gist_id}","starred_url":"https://api.github.com/users/jbowes/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jbowes/subscriptions","organizations_url":"https://api.github.com/users/jbowes/orgs","repos_url":"https://api.github.com/users/jbowes/repos","events_url":"https://api.github.com/users/jbowes/events{/privacy}","received_events_url":"https://api.github.com/users/jbowes/received_events","type":"User","site_admin":false},"created_at":"2018-07-16T14:12:44Z","updated_at":"2018-07-16T14:12:44Z","author_association":"MEMBER","body":"+1 on `ReleaseGroup`. I'd prefer the name `ReleaseStream` though. This is something captured in the stream stuff for PR support, but to handle major/minor/patch etc upgrades, having a proper concept for it would be very useful.\r\n\r\n-1 on moving this control from networkpolicy to microservice. having control over which bits serve traffic, and allowing multiple domains to point to the same vsvc is a big differentiator for hlnr.  is it actually a useful one? I dunno, but I like to think so :)\r\n\r\nThe microservice should handle garbage collection of vsvcs based on a configured number of live instances, the release streams in play, and the vsvcs that have network policies pointed at them. the networkpolicy should handle updating to a new vsvc, if configured to do so, based on vsvc readiness.\r\n\r\nTo support canarying and rolling deploys along with blue/green, a networkpolicy that updates can inform the vsvcs in play to adjust themselves.\r\n\r\nFor microservices that don't have an http component... I dunno. I think we can worry about that afterwards. maybe it looks like what you've outlined, or maybe its a new api object."}}
{"actor":{"display_login":"jbowes"}}