Faster Gitlab CI/CD pipelines
One improvement that would be nice in gitlab is an extension &#x2F; similar feature to the FF_USE_FASTZIP flag.<p>The default way to cache things is zip, which no matter the compression isn&#x27;t a great fit, you really want a format that is stream-able like tar. When its stream-able you can download the tar and unpack it at the same time, not waiting for the full download before beginning to unpack. Obviously you would compress the tar (normally) with lz4 or zstd or similar, using this approach you should generally see a reduction in total time to get and unpack the cache. node_modules even more so since it generally has zillions of small files, so the unpack time can be quite high.<p>Be nice if gitlab supported this approach OOTB.<p>Edit: An even better way if your environment supports it is to use a mountable image (esp for node_modules) for your caches, this basically removes the entire unpack phase of making the cache available, instead of unpacking you just mount the image and off you go.<p>In macOS this looks like a sparseimage via attached hdituil, or in linux a squashfs image with a writable overlay mount (you need a privileged container for this if in a container). Since the OOTB gitlab.com runners are a root-user linux VM (i think?) this approach should work quite well. SquashFS images are a great fit for node_modules especially as it moves the creation of the zillions of tiny files to cache-creation time rather than cache-use time. If you share the cache images via a hostPath mount or similar for existing images you can effectively make caches available in 0 seconds (just mount an already downloaded image and done).