It’s expected that the clone operation would have a performance impact, since you are creating new tensors. Since your model is tiny, you might see this overhead.
I would try to avoid the inplace operations and store the results of the torch.sigmoid calls into a new tensor instead of the a “placeholder”.