This snippet works fine.
aa = torch.FloatTensor(torch.randn(1,1,256))
bb = torch.FloatTensor(torch.randn(1,5332,256))
cc = torch.cat((aa,bb),1)
cc.shape
(1, 5333, 256)
type(cc)
<class ‘torch.Tensor’>
type(aa)
<class ‘torch.Tensor’>
type(bb)
<class 'torch.Tens
However this one  throws error :
embeddings = self.embed(captions)
pdb.set_trace()
embeddings = torch.cat((features.unsqueeze(1), embeddings), 1)
Debugging the type and shapes:
(Pdb) embeddings.shape
torch.Size([1, 5332, 256])
(Pdb) features.shape
torch.Size([1, 256])
(Pdb) type(embeddings)
<class ‘torch.Tensor’>
(Pdb) type(features)
<class ‘torch.Tensor’>
(Pdb) embeddings = torch.cat((features.unsqueeze(1), embeddings), 1)
*** RuntimeError: cuda runtime error (59) : device-side assert triggered at /opt/conda/conda-bld/pytorch_1524580938250/work/aten/src/THC/THCCachingHostAllocator.cpp:257
(Pdb)
Can’t understand why? Is it because one of the inputs can be a variable ?