As far as I can tell – I only had a very quick look at the paper – the LSTM is used as a decoder similar to Seq2Seq models.
The figure indicates that the output of FC is used as the first initial state which has a shape of (num_layers * num_directions, batch, hidden_size). Given that it’s used as a decoder num_directions=1 and usually num_layers=1 as well. So the expected shape is (1,B,128) So you can simple do unsqueeze(0) on the (B,128) tensor.
However, this clarifies only h_0 and c_0. To get the decoder going also requires some initial input. For decoding texts, that’s usually a special Start-of-Sequence <SOS> token.
Alternatively, the papers state “[…] initial input is obtained by converting output of the FC layer of size 4,096 into a vector of size 128 with another FC layer”. If this really is the input but not the initial hidden state, then they are sloppy with the terminology.
If all fails, just send an email to the authors to clarify. Usually works.