albanD:

You will need to derive the mathematical formula for what the gradient should be with pen and paper first.


In the research paper that I showed you, there is gradient equation (i.e. differentiation of Li w.r.t. Zi for every 'i’th feature if you compare with my code), but I don’t know why it was w.r.t. evey feature instead of w.r.t. parameters (weight and bias). Contradictorily, the loss formula doesn’t include use of parameters, but the features which go into the loss formula are result of mathematics of input and parameters!



 albanD:

Does it run out at the first iteration? Or after a while?
Can you try reducing the batch size to reduce memory pressure?


If I use smaller batch size then I could see results for first few batches, but I doubt that it would still go out of memory after some more iterations.
However, this loss function yields good results for bigger batch sizes. So, I tried with bigger batch size,but it went out of memory after some time without showing first result.