It really depends by the arch.
E.g. This was the position some years ago for the convolutional layers:
http://mipal.snu.ac.kr/images/1/16/Dropout_ACCV2016.pdf 1
More in general one interesting recent work co-authored by Google is autodropout but I don’t know why the code isn’t available:

  
      

      arXiv.org
  

  
    

AutoDropout: Learning Dropout Patterns to Regularize Deep Networks 1

  Neural networks are often over-parameterized and hence benefit from
aggressive regularization. Conventional regularization methods, such as Dropout
or weight decay, do not leverage the structures of the network's inputs and
hidden states. As a...


  

  
    
    
  

  



  

      github.com/google-research/google-research
  

  
    
  
	  
  

  
    
      Why cannot find the code for "AutoDropout: Learning Dropout Patterns to Regularize Deep Networks" ? 
    

    
      
        opened 
        
          
        
        Jun 7, 2021
      
      


      
        
          
          lcshr123
        
      
    

    
    
  


  
    Hi, I want to refer to the implementation of AutoDropout, but it seems that I ca…nnot find it here. The paper said that code is published in this website, so how can I touch it ?