There are multiple possible approaches and it depends how the activation shape is interpreted.
E.g. using [64, 512, 1, 28] you could squeeze dim3 and use dim4 as the “sequence” dimension (it’s one of the spatial dimension).
In this case, you could permute the activation so that the linear layer will be applied on each time step and permute it back to the expected shape for CTCLoss.
Something like this could work:
B, C, H, W = 64, 512, 1, 28
x = torch.randn(B, C, H, W)

x = x.squeeze(2) # remove H dim
x = x.permute(0, 2, 1).contiguous() # permute to [B, H, C=T]
T = x.size(1)

linear = nn.Linear(C, 76)
out = linear(x) # will be applied on each timestep (same as looping over dim1)
print(out.shape)
# > torch.Size([64, 28, 76]) # corresponds to [B, T, out_features]

criterion = nn.CTCLoss() # expects an input in [T, N=B, C]
out = out.permute(1, 0, 2).contiguous() # permute to [T, B, C]
out = F.log_softmax(out, dim=2) # create log probabilities

# create random target and lengths
target = torch.randint(1, C, size=(B, 10))
input_lengths = torch.full((B,), T, dtype=torch.long)
target_lengths = torch.randint(1, 10, (B,))

# calculate loss
loss = criterion(out, target, input_lengths, target_lengths)
loss.backward()