The new error could be caused by the mentioned matrix multiplication, since PyTorch tries to allocate 2GB, which would also be needed by this operation.
However, since this is expected you might want to reduce the memory usage e.g. by lowering the batch size.
As previously described, the initial error claiming to allocate 256GB is most likely caused by another operation and my best guess is that unwanted broadcasting might have been applied in an operation.