shivamsaboo17:

But the problem is, the patch values even after using sigmoid are either 0 or 1.


That is perhaps something about your method? I find that not terribly surprising if you optimize, you’ll have some gradient and then you push all things with one sign in the gradient towards 0 and the other towards 1. That is why the adversarial attacs usually say how much far they want to go in some norm (e.g. l-infinity aka max-norm for FGSA or l² or l¹ norm for some others).


And the coordinate prediction is always being done in the top left part of each image


That probably is related to how you generate your patch coordinates.
Best regards
Thomas