Self-Answering.
To use nn.CrossEntropyLoss, manipulated shape of output and label.
outputs.view(-1, outputs.shape[2]) # dim => (batch_size * max_seq_length) * num_tag
y_train.view(y_train.shape[0] * y_train.shape[1]) (batch_size * max_seq_length)

For example, let batch size is 32, max_seq_length = 80, num _tag = 18, then output shape will be (2560 * 18), label shape will be (2560, ).