Hi Xianshun!



 xianshunw:

However, this operation is not differentiable. Does someone know how I can calculate the bitwise > and still be able to backprop?


The greater-than operator (>) is basically a step function.
A common differentiable approximation to a step function
is a sigmoid.  (There are others.)
You could try:
c = nn.sigmoid (a - b)

You can sharpen (or smooth out) the sigmoid with a parameter:
c = nn.sigmoid (s * (a - b))

(By the way, are you sure you mean bitwise >?)
Good luck.
K. Frank