my intent was to append the unet layer just before the classification layer ,feed it the output of last layer before classfication layer, and give the output of the unet layer in the desired output channel number . I mistook 5124 as 512  …so that way i was wanting to give 256 input to conv layer ,get 256 from de conv  layer  so that is why put wrong output no of channels for the unet layers.
But overall objective was
Resnet50 minus classification  (5124,num classes)  -> Unet-> Classfication ( 512*4 ,num classes)
so have to set the dimension such in channel for deconv is fixed that will be i believe 2048 ,out can be 1024 ,similarly out from conv layer can be 1024 ,both together should total to 2048 for last Fc layer.