Based on the error message it seems that the manipulation of the model is indeed working, while the input concatenation is not:
RuntimeError: Given groups=1, weight of size 64 4 3 3, expected input[16, 3, 299, 299] to have 4 channels, but got 3 channels instead

This error indicates that the input is still using 3 channels, while 4 are expected.
Could you double check and/or post the code which creates the new input tensor?