title: Reliable events log (the not really Event Sourcing)
body: <p>I'm implementing a system that should store all the events in a consistent way, but at the same time I'd like to retain consistency by using sort of a mixed approach. Although the idea of event sourcing is pretty clear, - all the mutations go into ES log and consumers then create materialised views, what bugs me is exactly eventual consistency sine the system is now asynchronous system-wide.</p>
  
  <p>CQRS + ES approach suggests this should be resolved on UI level asking a client to wait. Suppose, I've got something close to what stackoverflow is. When I will have finish this question I'll hit "Post your question" button and the web-site will take me to my question straight away. With ES approach, hitting "Post your question" will mean that my question will be pushed into ES store to be processed later on and I will be seeing "We are publishing your question, hold on a while", right? That's what I want to avoid. I came up with the following schema:</p>
  
  <pre><code>CreateQuestionCommand command
      = new CreateQuestionCommand(uint userId, string questionBody, string[] tags)
  QuestionSavedEvent result
      = command.execute() // save to DB as usual, return events
  saveToES(result) // ugghhhhh...
  </code></pre>
  
  <p>If I ever forget <code>saveToES</code>, ES log becomes inconsistent and practically useless. If any of the devs on my team forgets it, the same thing - whole ES log is just a throwaway.</p>
  
  <p>Can this sort of an approach ever be viable? This would solve ES's drawback which is eventual consistency and still retain it's bright side which is always reliable events log.</p>
  
  <p>I came across some solutions.</p>
  
  <p><strong>1.</strong></p>
  
  <ul>
  <li>Some of the databases will allow access to commit log which can be a source of events, but this approach is a bit dull because reading commit log doesn't say anything about application business-logic which may be needed, i.e. you just read <code>UPDATE users SET loggedIn = "2018-12-30 10:00:00" WHERE id = 1</code> from the commit log, the event in this case is just <code>UserUpdated</code>, but on the application level it could be a much nicer <code>UserLoggedIn</code>.</li>
  </ul>
  
  <p>The other approach employed by ebay (allegedly, I've read it in one book on ES, won't recall the title, tho)</p>
  
  <p><strong>2.</strong></p>
  
  <ul>
  <li>Start transaction 1</li>
  <li><code>UPDATE domain.users SET loggedIn = "2018-12-30 10:00:00" WHERE id = 1</code></li>
  <li>Start transaction 2 (inner trx)</li>
  <li><code>INSERT INTO events.log SET event = "{name: UserLoggedIn, userId: 1}", timestamp = "2018-12-30 10:00:00"</code></li>
  <li>Commit transaction 2</li>
  <li>Commit transaction 1</li>
  </ul>
  
  <p>Then another process can poll this <code>events.log</code> to publish data into ES store.
  By design, each data mutator should be forced to return an <code>event</code> and a pointer to <code>transaction 1</code> and this is how the system will never "forget" to commit event to the database.</p>
  
  <p>The third approach is to use 2PC (two phase commit) which requires distributed transaction manager, and 2PC support by both the message broker and the database, and because of this constraints I don't even look into this direction.</p>
  
  <p>So, I just wonder, in your experience what is the best way to keep the system to be "old-fashioned" (synchronous), but guarantee reliable events log at the same time? (Yeah, I want all the goodies from both of the worlds :) )</p>
  
  <p><em>Disclaimer</em>: I know how bloated the question is, if you think this question doesn't belong in the community because of this, let me know, I will take it down.</p>