Kafka, GDPR and Event Sourcing
I&#x27;m not a Kafka user. However, I am a long-time keen implementer of event sourcing and log-derived data stores. Many of these involve per-user or per-individual sqlite databases.<p>I cannot understand why Kafka is causing people problems. I am surprised that this seems to be being accepted as a problem with event sourcing in general.<p>The most obvious solution I came up with with GDPR right to be forgotten was to have per-user logs. User gone? Delete their log.<p>Then you just need to handle how you share the projected data from the individual event logs. Messages are unlikely to be spread over multiple log events and can be just sourced directly. It&#x27;s clear who owns that data: it&#x27;s the user who wrote the message as they can at any time delete the only copy.<p>Aggregation is more difficult but can be done if you structure the things you are aggregating such that you only need a few lookups per user. Again you can cache references to certain events that can be efficiently looked up using a (user_id, event_id) pair. There&#x27;s nothing to stop you storing the latest projection in a separate table of the per-user database file.<p>Is there some technical reason why so many Kafka users seem to be having such difficulty with this?