Redis Streams and the Unified Log
The unified log pattern - and event sourcing based on that - is a very powerful pattern.We are doing some work at my job right now inspired by this approach (in no small part, that LinkedIn article). I hope we can open-source some of this tech, or at least start blogging about the concepts as nobody really talks about it, and the mechanics are a little tricky to get right.We currently think the best approach is to bring ideas from DDD (specifically the idea of a bounded context and a domain), a unified log which we call the GEL (for Global Event Log), domain-specific event logs, event-sourcing with CQRS and then some quite lightweight means to produce "projections" (different structural representations of data).We therefore have a few pieces that get names and become easy to reason about: every event goes to the GEL. Domains have systems that listen to the GEL for events they are interested in and map those events into commands on a command service. Clients can issue commands directly, as well. Commands alter projections and produce events back to the GEL. Clients and domains can query a domain's query service.It's not really rocket science, these ideas are as old as the oldest mainframe you've ever heard of, but it affords incredible flexibility and unrivalled performance.We're using Kinesis for our GEL that has a read performance issue, but not a big enough issue for us yet to look at alternatives - the zero-admin cost is attractive to us right now.For me, now, MVC and CRUD style applications look like really odd anti-patterns. I'm amazed we got as far as we have with them.