I actually solved this problem. My fault for not showing my FCN model code. The problem was that I had batch normalization between the convolution layers. This created discontinuities between the patches when stitching the images back together.