Storage Optimization, Software Architecture, AWS, Icebergs, Kafka.
for the 99th percentile. This brings us to the hard deletion problem, which means at some point not very far in the future, we have to make sure all the traces associated with these deleted files are removed from our system. This includes cleaning up bytes from S3, deleting any indexing done on it