{"action":"created","issue":{"url":"https://api.github.com/repos/cdodev/design/issues/1","repository_url":"https://api.github.com/repos/cdodev/design","labels_url":"https://api.github.com/repos/cdodev/design/issues/1/labels{/name}","comments_url":"https://api.github.com/repos/cdodev/design/issues/1/comments","events_url":"https://api.github.com/repos/cdodev/design/issues/1/events","html_url":"https://github.com/cdodev/design/issues/1","id":139897377,"number":1,"title":"Event stream: Really a golden source?","user":{"login":"k0001","id":3302,"avatar_url":"https://avatars.githubusercontent.com/u/3302?v=3","gravatar_id":"","url":"https://api.github.com/users/k0001","html_url":"https://github.com/k0001","followers_url":"https://api.github.com/users/k0001/followers","following_url":"https://api.github.com/users/k0001/following{/other_user}","gists_url":"https://api.github.com/users/k0001/gists{/gist_id}","starred_url":"https://api.github.com/users/k0001/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/k0001/subscriptions","organizations_url":"https://api.github.com/users/k0001/orgs","repos_url":"https://api.github.com/users/k0001/repos","events_url":"https://api.github.com/users/k0001/events{/privacy}","received_events_url":"https://api.github.com/users/k0001/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2016-03-10T14:10:37Z","updated_at":"2016-03-10T16:10:09Z","closed_at":null,"body":"Regarding the following:\r\n\r\n> The main difference of a functional app is that we don't mutate state directly. In fact current state is derived entirely from events that have been observed to happen, so **we should be able to recover any state by replaying the events we have seen**. The stream of events that has occurred is the golden source of data in the system and can be folded down into any other data source to express state.\r\n\r\nI am hesitant about the part that I highlighted. In theory, that is true. In practice, recovering events that were saved sometime in the past is incredibly hard, as we already know from our experience at Fynder. There are some workarounds such as keeping state snapshots, or attempting to have some kind of migrations for events themselves, but these are just ad-hoc kludges and they can't really be generalized in a reliable manner. \r\n\r\nI suggest we leave out the idea of “recovering any state by **replaying** the events we have seen”. We should have events, yes, but consider them “messages” as in a “message passing”, and use that to drive changes to the UI in real time (say, through a PubSub API over WebSockets), but those messages are never stored in a database, they just exist in memory and distributed over the network until they are not needed anymore.\r\n"},"comment":{"url":"https://api.github.com/repos/cdodev/design/issues/comments/194928508","html_url":"https://github.com/cdodev/design/issues/1#issuecomment-194928508","issue_url":"https://api.github.com/repos/cdodev/design/issues/1","id":194928508,"user":{"login":"boothead","id":87764,"avatar_url":"https://avatars.githubusercontent.com/u/87764?v=3","gravatar_id":"","url":"https://api.github.com/users/boothead","html_url":"https://github.com/boothead","followers_url":"https://api.github.com/users/boothead/followers","following_url":"https://api.github.com/users/boothead/following{/other_user}","gists_url":"https://api.github.com/users/boothead/gists{/gist_id}","starred_url":"https://api.github.com/users/boothead/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/boothead/subscriptions","organizations_url":"https://api.github.com/users/boothead/orgs","repos_url":"https://api.github.com/users/boothead/repos","events_url":"https://api.github.com/users/boothead/events{/privacy}","received_events_url":"https://api.github.com/users/boothead/received_events","type":"User","site_admin":false},"created_at":"2016-03-10T16:10:09Z","updated_at":"2016-03-10T16:10:09Z","body":"Here's a motivating example:\r\n\r\nAt a hedge fund I worked at we had a very event driven model. A schedule would kick off a number of actions in sequence:\r\n\r\n * Fetch market price\r\n * Run algo (which by the way was also just a fancy scanl)\r\n * Scale signal by sizes of accounts to generate desired trades\r\n * Electronically execute trades (also event driven - but a separate system)\r\n * Allocate filled trades\r\n * Archive completed trade record\r\n\r\nWe modeled this as passing a token (called a ticket) around that had information added to it at each stage (update statements pretty much) until we ended up with a full ticket at the end. This was the equivalent of throwing all the events away after folding over them.\r\n\r\nThis worked fine, until we wanted to to do some analysis on how long it took to get a ticket between two point (or fold the events a different way). Because we didn't keep all the actual events that happened, we were reduced to grepping log files pretty much. If we had architected our system as a fold over events, then we would have had to simply write a different fold to get the answers we needed.\r\n\r\nThis is why I consider the event stream the golden source: because it contains information that it valuable, even if you only realise that value at a later date. Also keeping all the events around doesn't preclude optimizations like folding forward from the last snapshot."}}