My tensor cores wont work for anything not multiple of 8 ? I thought that they are triggered on their own. And for the cudnn is there is a certain command to trigger it to use the tensor cores or just multiple of 8 ?
To check if i understand this correctly , if i make the embeddings 256 instead of 300 , that means that all the conv layers are 3x256,4x256 … which is a multiple of 8 , is this correct ? And does this contribute in the speedup ?