Hi @Sajjad_Hossain, Are you considering to apply attribution algorithms to the outputs with respect to the inputs in order to compute importance scores ? What kind of importance are you’re looking for ?
We have a library for model interpretability that allows to attribute the output target to the inputs or neurons of the network. Is that something that can be useful for you?

  
      
      GitHub
  
  
    

pytorch/captum 13

Model interpretability and understanding for PyTorch - pytorch/captum