Just ran an experiment checking a commit before the recent LLVM bump on a newly created disk cache:
...$ git checkout f696b566f40baa87d776c92a12b03cca1d83bfd1
...$ bazel clean --expunge
...$ time bazel build --disk_cache=~/bazel_cache //tensorflow/tools/pip_package:build_pip_package
...
INFO: Elapsed time: 1176.812s, Critical Path: 251.94s
INFO: 11114 processes: 1289 internal, 9825 local.
INFO: Build completed successfully, 11114 total actions

real    19m36.954s
user    0m0.359s
sys     0m0.380s

Next, switch to the LLVM bump and compile again:
...$ git checkout a868b0d057b34dbd487a1e3d2b08d5489651b3ff
...$ time bazel build --disk_cache=~/bazel_cache //tensorflow/tools/pip_package:build_pip_package
...
INFO: Elapsed time: 523.303s, Critical Path: 208.88s
INFO: 3273 processes: 166 internal, 3107 local.
INFO: Build completed successfully, 3273 total actions

real    8m43.377s
user    0m0.166s
sys     0m0.171s


Note, this is on a fast machine, OSS contributors likely won’t have access to these specs:
...$ cat /proc/cpuinfo    # 56 CPUs
...
processor       : 55
vendor_id       : GenuineIntel
cpu family      : 6
model           : 79
model name      : Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz
stepping        : 1
microcode       : 0xb00003e
cpu MHz         : 3192.602
cache size      : 35840 KB
...
...$ free -h              # 126 GB RAM
               total        used        free      shared  buff/cache   available
Mem:           125Gi        42Gi        61Gi       1.5Gi        21Gi        80Gi
Swap:          127Gi       741Mi       127Gi

This experiment could be repeated for other LLVM bumps (or TFRT ones too).
So, the most recent LLVM bump caused a recompile of almost 50% (time-wise) of a full compile. This impacts anyone developing in OSS but also prevents us from using caching and GitHub Actions for GitHub presubmits. Maybe both of our teams can invest some time to reduce this (assuming this happens on other LLVM bumps)?