Thanks for the reply!
i made it print out the shape of created tensors everytime function executes in my program and it goes like this:
torch.Size([1, 3, 299, 299])
torch.Size([1, 3, 299, 299])
torch.Size([1, 3, 299, 299])
torch.Size([1, 3, 299, 299])
torch.Size([1, 3, 299, 299])
torch.Size([1, 1, 299, 299])
Traceback (most recent call last):
  File "main.py", line 11, in <module>
    main()
  File "main.py", line 7, in main
    game.game()
  File "/home/adasko147/Desktop/VSCode/SZISMIECIARKA/game.py", line 111, in game
    adamB.predict(smiec)
  File "/home/adasko147/Desktop/VSCode/SZISMIECIARKA/uczenie_adamB.py", line 117, in predict
    outputs = net(pil_to_tensor)
  File "/home/adasko147/Desktop/VSCode/SZISMIECIARKA/venv/lib/python3.6/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/adasko147/Desktop/VSCode/SZISMIECIARKA/uczenie_adamB.py", line 44, in forward
    x = self.pool(F.relu(self.conv1(x)))
  File "/home/adasko147/Desktop/VSCode/SZISMIECIARKA/venv/lib/python3.6/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/adasko147/Desktop/VSCode/SZISMIECIARKA/venv/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 349, in forward
    return self._conv_forward(input, self.weight)
  File "/home/adasko147/Desktop/VSCode/SZISMIECIARKA/venv/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 346, in _conv_forward
    self.padding, self.dilation, self.groups)
RuntimeError: Given groups=1, weight of size [6, 3, 5, 5], expected input[1, 1, 299, 299] to have 3 channels, but got 1 channels instead

It looks like it has 1 for the first dim and 3 channels too. At some point channel changes from 3 to 1. Maybe it is something with provided images. Im not really sure what to do now