XLNet bidirectional input pipeline requires batch size at least 2
This may not be a true bug since it's mentioned in the paper that 

> each of the forward and backward directions takes half of the batch size

but when using the bidirectional input pipeline, any call to `XLNetModel.forward()` will raise an error of the form 

```
RuntimeError: shape '[x, y, z]' is invalid for input of size 0
```

if the **batch size of the `input_ids` passed is less than 2**. This is because it halves (integer div) `bsz` in accordance with the above quote in the following block:

```
if self.bi_data:
            fwd_pos_seq = torch.arange(beg, end, -1.0, dtype=torch.float)
            bwd_pos_seq = torch.arange(-beg, -end, 1.0, dtype=torch.float)

            if self.clamp_len > 0:
                fwd_pos_seq = fwd_pos_seq.clamp(-self.clamp_len, self.clamp_len)
                bwd_pos_seq = bwd_pos_seq.clamp(-self.clamp_len, self.clamp_len)

            if bsz is not None:
                fwd_pos_emb = self.positional_embedding(fwd_pos_seq, inv_freq, bsz//2)
                bwd_pos_emb = self.positional_embedding(bwd_pos_seq, inv_freq, bsz//2)
```

The result is a batch size of `0`, which obviously wreaks havoc later on. It's only relevant if people are trying to run a MWE with really small input as I was for testing, but maybe an assert statement somewhere near the top of the `XLNetModel.forward()` function is a good idea, conditional on `bi_data` being `True`. 

More generally, a shape mismatch is caused for the same reason if `bi_data` is `True` and `bsz` is any positive odd integer. 
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
