I’ve launched again the Tensorflow Addons (WIP) Github Action CI with these images at:

  

      github.com/tensorflow/addons
  

  
    
  
    
  

  
    
      [WIP] Testing new build image
    

    
      tensorflow:master ← seanpmorgan:new-build-image
    

    
      
        opened 
        
          
        
        Jul 1, 2021
      
      

      
        
          
          seanpmorgan
        
      

      
        
          +2
          -3
        
      
    
  


  
    # Description

Just testing the py37 build
  

  

  
    
    
  

  





 angerson:

Needs decision: custom-op functions for SIG Addons, IO, etc. (I want to get backing from leadership to guarantee support)


Currently we had some issues cause custom-ops images are unmaintained in TF. See more at:

  

      github.com/tensorflow/addons
  

  
    
  
    
  

  
    
      Use multipython image for dev container
    

    
      tensorflow:master ← seanpmorgan:new-dev-base
    

    
      
        opened 
        
          
        
        Nov 9, 2021
      
      

      
        
          
          seanpmorgan
        
      

      
        
          +3
          -3
        
      
    
  


  
    # Description

Previous container only supports up to py3.6 which TF2.7 is not… built for. TF team has stopped publishing these custom-op images since TF2.5.

Using the CUDA image makes the container larger, but it still uses the cpu only version of TF so it's accurate to call it dev_cpu. Building the gpu image would be pretty trivial now, but also this limits the maintenance burden on our team.

There is no multipython container without GPU installs:
https://console.cloud.google.com/gcr/images/tensorflow-testing/GLOBAL
  

  

  
    
    
  

  





 angerson:

Here’s a parallel topic for SIG Build contributors we can also discuss here:


What I see is that with these images we are loosing the small size runtime and devel images if we compare these with the current “officially” published  images (~ 400/600MB) as the GPU/CUDA layers are not optional.
I suppose/hope that we want to propose these images also as a reproducible environment to prepare, compile, lint and test PRs for our community contributions. As with these image the envs could be as close as possible to that we use in the TF CI (if we use these images) there is some hope that we could be almost exactly on the same page.
If this is the case I suppose that having also low overhead images, when you want to contribute a quick PR, could be still valuable. Often you don’t need to have a GPU or consume a large size GPU images to contribute to TF or you want to just minimize the cloud costs and boostrap waiting time when you are asking resources to the cloud for a new development env. e.g. Github Codespaces or Kubernetes devcontainer POD.
As a side note I still hope that we could integrate pre-commits hooks from:
  

      github.com/tensorflow/tensorflow
  

  
    
  
    
  

  
    
      Update TF official devel images to Ubuntu 20.04 and precommit hooks
    

    
      tensorflow:master ← bhack:devel_ubuntu_20.04
    

    
      
        opened 
        
          
        
        Apr 7, 2021
      
      

      
        
          
          bhack
        
      

      
        
          +79
          -17
        
      
    
  


  
    Update TF official devel images to Ubuntu 20.04 LTS and install pytlint.

`ci_…build.sh` is currently broker for `ci_sanity.sh` caused by a misalignment  between CI sanity and local images on developer machine (See https://github.com/tensorflow/tensorflow/issues/47989).
Now that https://github.com/tensorflow/tensorflow/pull/48291 is merged this will let developer/contributors to locally execute python linting steps in the docker devel container with:
```
docker run --rm -it -v $PWD:/tensorflow -w /tensorflow tensorflow/tensorflow:devel tensorflow/tools/ci_build/ci_sanity.sh --pylint --incremental
```

It could be nice if later the full CI sanity could be run exactly on the same image where developer are working every day to avoid other potential misalignment.

If this works we could change the documentation for linting. 

I could expand this PR or with a new one to introduce other steps (e.g. include `--clang_format` ?) so that we could have a working minimal configuration that could be also contributed in the repository as a  pre-commit hook  considering that the the whole `ci_sanity.sh` is too heavy to be run as a git hook.

/cc @mihaimaruseac @angerson
  

  

  
    
    
  

  


We are full of multiple formatting request comments in PRs that requires manual intervention. I hope that we could enforce these a little bit more on the local dev machine with pre-commits to lower the linting roundtrips  comments on the PR itself and CI run cycles.
I hope that these new reproducible env will enable a read only cache sharing from Tensorflow master so that we could quickly build TF in these envs with a reasonable amount of time on our local dev hw or cloud resources when we want to contribute a PR to TensorFlow.