Event logs · DDD · RDBMS

Event logs, logical vs physical

I’m often loathe to blog about things I have read in a book, for fear of plagiarising, and not adding to the discourse, but something that I have read really needs to be shared more and more.

The reason I am so excited is because this answers a difficult question, how to deal with undo.

When an event log is created (in a database), a decision is normally made, are the events that are being appended to the log this the action that is being taken (this is the logical event log) or the state after the action.

A bank statement is an example of an event log that holds both state (the balance) and logic (what occurred to change the state)

However, the cost of storing the state can be quite a bit, as there might be a lot of data in each state, and, further, some of it may never be used again.

The cost of storing only the actions that change the state is the ability to “undo” is expensive in time and CPU effort. How? An action can only be performed on a state, the reciprocal action (the undo) may not be simple to calculate. For example, “Delete this data block” requires knowledge of the previous state (what was deleted) in order to rebuild that data block.

The only way to “undo” using the logical log is often achieved using the following steps.

  • Snapshots of the state are taken every n events
  • When an “undo” is requested, load the state at that snapshot
  • Then replay the events on that snapshot from the time of snapshot to the step before the current one.

Wash, rinse, repeat for each “undo”.

A far simpler algorithm exists for undoing a physical log, change the viewer from now to the previous state.

A database has a long lived projection but the WAL (which is the event log) is temporal, effectively the length of time between snapshots, meaning the state of the system before a previous snapshot is unreachable.

In an event driven system, though, the aggregates are normally only a few events long, and the history of the state may be required for several years, meaning that the states need to be persisted for a great deal of time.

Understanding this balance, like everything in engineering, allows developers to make informed decisions on what approach (or hybrid approach) to take.

Published:
comments powered by Disqus