Hi, thanks for the reply.
I haven’t found any particular code in the keras call function call, compute_attention 1 that makes some underhood 1d transformation (at least that I can identify). This goes a bit beyond my understanding, I havent been able to find any 1D attention examples. Do you know if issues in pytorch repo can be use to ask how to do calculations like this?