Fair enough :)! I just skimmed through your post on my phone :).
Hm, difficult what to say. I’m not even sure what your trying to do. When you say that the captions tensor shape is [5, 14, 256] I assume that means [batch_size, seq_len, embed_dim] – i.e., that’s the tensor after the embedding.
Before the unsqueeze(), your feature tensor has a shape of [5, 15000], again assuming 5 is the batch size. That means you have 15k features for a data sample. Why would you want to concatenate them with the embeddings before the LSTM? Where is the notion of a sequence w.r.t. to the features.
I would expect the push the captions through the LSTM, get the last hidden state of shape [batch_size, embed_dim] (i.e., [5, 256]) and then concatenate them with the original feature tensors of shape [5, 15000] to get a tensor of shape [5, 15256].
But again, I don’t know the task and what the data is, so I’m only guessing here.