Storage Optimization, Software Architecture, AWS, Icebergs, Kafka.
are generating data, contributing to about 2 Petabytes of data created per week. This is a complex mix of data coming from a variety of sources, including text, images, image sequences, large sources like IMFs¹, and generated proxies and mezzanines. The sheer volume is staggering, making it