Hi guys,
Wondering if anyone has read the new paper from the Alexa team regarding BERT size reduction.

  
      
      arXiv.org
  
  
    

Optimal Subarchitecture Extraction For BERT 8

We extract an optimal subset of architectural parameters for the BERT
architecture from Devlin et al. (2018) by applying recent breakthroughs in
algorithms for neural architecture search. This optimal subset, which we refer
to as "Bort", is...


  
  
    
    
  
  



  
      
      GitHub
  
  
    

alexa/bort 8

Repository for the paper "Optimal Subarchitecture Extraction for BERT" - alexa/bort


  
  
    
    
  
  


If anyone has any thoughts on it or would like to discuss please comment here.
Thanks