Hello, @vainaijr, I am running into the same issue. Could you possibly just transpose the tensor from (batch, feature, seq_len) —> (batch, seq_len, features)? then feed that through layernorm, and then re-transpose back? Would this work?