Hi,
The definition of continutiy: The function  f  is  continuous at some point   c  of its domain if the limit of  f ( x ), as  x  approaches  c  through the domain of  f , exists and is equal to  f ( c ).

In detail this means three conditions: first,  f  has to be defined at  c  (guaranteed by the requirement that  c  is in the domain of  f ). Second, the limit on the left hand side of that equation has to exist. Third, the value of this limit must equal  f ( c ).
So, ReLU etc all are continuous.
Here is a great post 6 by @rasbt .
In the end, I think you have confused non-linearity and discontinuity.



 bolt25:

Shouldn’t the non-linearity create a problem while calculating gradients?


Actually, all we are looking for is to have non-linear activation functions to break linearity between each W.x+b calculations in different layers.
Bests