For anyone wondering I got this to work by flattening the batch and sequence length (T) dimensions:
# x.shape = [192,8,5]
y = torch.flatten(x, start_dim=0, end_dim=1).to(torch.int64)
# y.shape = [1536,5]
bert_embeds = bert(y)