The blocky or patchy approach is not a pytorch issue, from my understanding it is caused by one of the two reasons, either your network does not have enough context to work with i.e. input block size is too small or your code to stitch them back together is messing up somehow.
The code mentioned in the lined question of @farazk86 should work well for stitching the patches back together without any block artefacts.  You should have a similar code, copied here for easy reference:
def unpad(patch, index, shape, pad_width=4):
    """
    Remove `pad_width` voxels around the edges of a given patch.
    """

    def _new_slices(slicing, max_size):
        if slicing.start == 0:
            p_start = 0
            i_start = 0
        else:
            p_start = pad_width
            i_start = slicing.start + pad_width

        if slicing.stop == max_size:
            p_stop = None
            i_stop = max_size
        else:
            p_stop = -pad_width
            i_stop = slicing.stop - pad_width

        return slice(p_start, p_stop), slice(i_start, i_stop)

    D, H, W = shape

    i_c, i_z, i_y, i_x = index
    p_c = slice(0, patch.shape[0])

    p_z, i_z = _new_slices(i_z, D)
    p_y, i_y = _new_slices(i_y, H)
    p_x, i_x = _new_slices(i_x, W)

    patch_index = (p_c, p_z, p_y, p_x)
    index = (i_c, i_z, i_y, i_x)
    return patch[patch_index], index

If stitching is not your problem them look into increasing the block size.
P.S. I;m very interested in your mentioned approach:

I use a combined loss of weighted DICE and weighted CE

Can you link this being used somewhere I would like to see it in action. or explain the implementation
Thanks