Khawar_Islam:

My code. Is the conceptually same for the only stage 0?


I don’t believe so, since the original code is not using pos_embed at all or is code missing?



 Khawar_Islam:

Which one is correct? x = x + self.pos_embed1? x = self.pos_embed1


I’m not familiar with your code and use case, but note that:
        x, H, W = self.patch_embed1(x)
        x = self.pos_embed1

would override the first returned x, so you should check, if that’s indeed the desired use case or of you want to add both values together.