@shakeel608 Have you done your task ?
I am using a transformer network for my audio, ofcourse the encoder part only for multihead attention using key quey and value matrices.
Could you plz explain what is the purpose of this H at last? Is this only for rewighted H, for better classification.
Regards