Could you explain a bit how you are calculating the “gradient of the cost” and how the gradients should be calculated?
Based on the description it seems you would like to calculate the 2nd derivative?