Hello vainaijr!



 vainaijr:

Is there any problem with an entirely binary neural network


There is no conceptual problem with such a network, although I
don’t see what advantage one would have (other than, perhaps,
economy of storage).
I suppose one might view a binary network as being a better
approximation to some particular biological or physical system.


what would be the substitute for gradient descent be in such a case,


There’s the rub.  This is a binary (special case of an integer)
optimization problem.  (See integer programming.)  This is known
to be hard.  There are various (inexact) optimization algorithms you
could use – branch-and-bound, simulated annealing – but training
will be expensive, and likely prohibitively so for networks of any
significant size.
Differentiability and gradient descent are your friend.
Good luck.
K. Frank