Microsoft Fabric, Data Engineering, Data Science, Python.
Parquet files containing taxi trip data to evaluate the performance of libraries when handling multiple files. I uploaded 5 files, totaling 250 MB and containing 160 million rows, to the MS Fabric Lakehouse. Given that Pandas doesn’t support multi-file reads, I excluded it from this comparison.