I’ve also tried to make a build with the CPU receipt + remote cache inside these images on the commit f3c361931fe449302953596e8f99fcd737b9390c (master):
bazel --bazelrc=/usertools/cpu.bazelrc build --config=sigbuild_remote_cache tensorflow/tools/pip_package:build_pip_package
I already see that we are having many cache misses. How frequently the remote cache is updated? Is it only with CI jobs orchestration for nightly release?
I cannot see the orchestration script-scheduling as for Github Actions but I suppose that if we are only going to update it on nightly we will be strongly conditioned by:

  
    
    
    LLVM updates and bazel cache Build
  
  
    As we are updating LLVM two times at day I’ve tried to query bazel with: 
bazel aquery  "rdeps(//tensorflow:*,//third_party/llvm:*)" --include_aspects 2>/dev/null | grep Compiling | wc -l 
I am not a bazel ninja so probably the query could be wrong or improved but I currently see 9938 files on master (CPU only). 
What is the effect of this bi-daily rolling update on the average community contributor compiling workflow/environment and his bazel cache?
  


Is It will plausible to schedule your internal job (I mean the the on without --remote_upload_local_results=false) on every master commits or at least after any llvm updates/sync?
If not:

can we publish somewhere the updated nightly master commit so we know on which commit the remote cache is positioned?
Is it safe enough to contribute a PR starting from the “nightly commit”?

Edit:
Just the last point from the first quick pass.
For a developer image or if we want to derive an official developer image it is quite annoying to have root permission files created inside the container on your host mounted TF source path when you are back on the host. Probably It could be ok if you are always going to create a temporary source code checkout or if we suggest to maintain the TF  source in a named volume but I suppose this will not be the main use case.
So we already discussed this on the Build repository some months ago and now we have also introduced a default user in official Keras developer image.
We don’t have too much upstream solutions so I think we could introduce a default user.