Hi again !
Here is an explanation of my use-case :
I have one-hot encoded sequences which are L x 4 in dimensions (L being the lenght of the sequence).
They look like this :
[[1,0,0,0],
[1,0,0,0],
[1,0,0,0],
[0,1,0,0],
[0,0,1,0],
[1,0,0,0],
[0,0,0,1],
[1,0,0,0]] (just a silly example).
These sequences are associated with a value that I am trying to predict.
They have various sizes, and I’ve tried 0 padding them before (which works) but I am afraid this will lead to some problems because we are looking at aligned motifs. (example a [1,0,0,0] followed by a [0,0,1,0] that are always at the same position). I am afraid that padding would remove this information.
As for a mask, I’ve not thought of this, do you have any general idea how it could be implemented so that the network “cuts” the padded part of the sequences ?
Edit : For now I’ve gone around the problem by using a batch_size of one, but it greatly increases computing time, which is going to be a burden when I try to scale this up to datasets of over 1,2 or 3M sequences.
Thank you very much for your answers,