Great thanks 
I currently use it like this:
        self.optimizer.zero_grad()
        loss = self.criterion(log_action_probabilities, rewards)
        loss.backward()
        # clipping to prevent nans:
        # see https://discuss.pytorch.org/t/proper-way-to-do-gradient-clipping/191/6
        torch.nn.utils.clip_grad_norm_(self.parameters(), 5)
        self.optimizer.step()
        self.log_action_probabilities.clear()
        self.rewards.clear()

With the clipping I try to get rid of this error:

    action_idx = distribution.sample()      #
  File "/home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages/torch/distributions/categorical.py", line 107, in sample
    sample_2d = torch.multinomial(probs_2d, 1, True)
RuntimeError: invalid multinomial distribution (encountering probability entry < 0)

Is this the right case / way to use the clipping?
Does anyone know other methods of handling the above error such as: Using a different network (did not help for me), using other optimizers, other loss functions?
My current problem is that without a lr of lower than 0.1 my algorithm seems to learn nothing. But with this high lr I get the above error. Clipping in the way above does not help:
    def forward(self, state: torch.tensor, legalCards: torch.tensor):
        state = state.resize_(180)
        probs = self.network(torch.FloatTensor(state))
        probs = probs * legalCards
        distribution = Categorical(probs)
        print(probs)
        print(distribution)            #
        action_idx = distribution.sample()      #
        log_action_probability = distribution.log_prob(action_idx)
        self.log_action_probabilities.append(log_action_probability)

Will produce this output after some time:
tensor([0.0000e+00, 0.0000e+00, 0.0000e+00, 0.0000e+00, 0.0000e+00, 0.0000e+00,
        0.0000e+00, 0.0000e+00, 0.0000e+00, 5.1009e-16, 0.0000e+00, 0.0000e+00,
        0.0000e+00, 0.0000e+00, 0.0000e+00, 0.0000e+00, 0.0000e+00, 0.0000e+00,
        0.0000e+00, 0.0000e+00, 0.0000e+00, 0.0000e+00, 0.0000e+00, 0.0000e+00,
        0.0000e+00, 0.0000e+00, 5.0108e-22, 0.0000e+00, 0.0000e+00, 0.0000e+00,
        0.0000e+00, 0.0000e+00, 0.0000e+00, 0.0000e+00, 0.0000e+00, 0.0000e+00,
        0.0000e+00, 0.0000e+00, 0.0000e+00, 0.0000e+00, 0.0000e+00, 0.0000e+00,
        0.0000e+00, 0.0000e+00, 0.0000e+00, 0.0000e+00, 0.0000e+00, 0.0000e+00,
        0.0000e+00, 0.0000e+00, 0.0000e+00, 0.0000e+00, 0.0000e+00, 0.0000e+00,
        0.0000e+00, 0.0000e+00, 0.0000e+00, 0.0000e+00, 0.0000e+00, 0.0000e+00],
       grad_fn=<MulBackward0>)
Categorical(probs: torch.Size([60]))



tensor([nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
        nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
        nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan],
       grad_fn=<MulBackward0>)
Categorical(probs: torch.Size([60]))


Maybe the problem is that probs goes to zero…