Storage Optimization, Software Architecture, AWS, Icebergs, Kafka.
Collector (GC) Workers listening off of this Kafka topic, and perform deletion on each row. These are horizontally scalable and autoscaled based on any spikes or backlog in the Kafka pipeline. The GC workers take care of the complete cleanup of an object. It first deletes all bytes in S3. This can