title: Gradient descent implementation
body: <p>I've implemented both the batch and stochastic gradient descent. I'm experiencing some issues though. This is the stochastic rule:</p>
  
  <pre><code>1 to m {  
   theta(j):=theta(j)-step*derivative (for all j)  
  } 
  </code></pre>
  
  <p>The issue I have is that, even though the cost function is becoming smaller and smaller the testing says it's not good. If I change the step a bit and change the number of iterations, the cost function is a bit bigger in value but the results are ok. Is this an overfitting "symptom"? How do I know which one is the right one? :)   </p>
  
  <p>As I said, even though the cost function is more minimized the testing says it's not good. </p>