Back in January 2021, the Architecture and Data Engineering teams hosted a 3 part book club on Ben Stopford's [[ https://www.confluent.io/wp-content/uploads/confluent-designing-event-driven-systems.pdf | Designing Event Driven Systems ]] ([[ https://docs.google.com/document/d/1LW4WapYPVu9u2dz4HJJFay5nnoRyVwE22oiIP-0qrR0/edit#heading=h.mudi298cwkxk | notes ]]).
In September 2021, the Architecture and Data Engineering teams hosted a watch party and discussion of Martin Kleppmann's [[ https://www.confluent.io/blog/turning-the-database-inside-out-with-apache-samza/ | Turning the Database Inside Out ]] talk ([[ https://docs.google.com/document/d/1JohgDFQ8YB3Em2KCbv9eC5yuyiWxL4qYsKdSbzleSsw/edit | notes ]]).
In spring of 2021, we had a meeting or two in a 'data infrastructure working group' where we talked about some of these ideas ([[ https://docs.google.com/document/d/1CpxSbL1RfCfnSnl2tFrMLoLFx_Dd2I4zRzlaM1qjcCw/edit# | notes ]])
If weren't able to make these past events, or you are new to some of these ideas, [[ https://wikitech.wikimedia.org/wiki/Event_Platform#Background_reading | here's a list of background reading ]] you could check out.
Both events started some interesting discussions. We'd like to continue those discussions more asynchronously so we can have time to deep dive into the ideas. This ticket will be used as a discussion forum for interested parties.
I'll copy some of the higher level discussion ideas from the discussion notes from those sessions, and we can expand on them in the comments as we see fit.
---
- Event Sourcing / Events as source of truth
- Turning the database inside out
- How do deal with eventual consistency?
- Event Driven Systems to solve runtime decoupling and data integration problems
- Event Driven Systems to solve organizational problems
- "Data on the outside" as first class citizen
- CQRS and single writer principal
- What event driven systems do we have at WMF now?
- What would parts of MediaWiki look like we if were to redesign them as event driven?
- ...
//NOTE: This ticket is not meant to reach any conclusions. It is meant for educational and discussion purposes.//