title: Cost function in logistic regression gives NaN as a result
body: <p>I am implementing logistic regression using batch gradient descent. There are two classes into which the input samples are to be classified. The classes are 1 and 0. While training the data, I am using the following sigmoid function:</p>
  
  <pre><code>t = 1 ./ (1 + exp(-z));
  </code></pre>
  
  <p>where</p>
  
  <pre><code>z = x*theta
  </code></pre>
  
  <p>And I am using the following cost function to calculate cost, to determine when to stop training. </p>
  
  <pre><code>function cost = computeCost(x, y, theta)
      htheta = sigmoid(x*theta);
      cost = sum(-y .* log(htheta) - (1-y) .* log(1-htheta));
  end
  </code></pre>
  
  <p>I am getting the cost at each step to be NaN as the values of <code>htheta</code> are either 1 or zero in most cases. What should I do to determine the cost value at each iteration? </p>
  
  <p>This is the gradient descent code for logistic regression:</p>
  
  <pre><code>function [theta,cost_history] = batchGD(x,y,theta,alpha)
  
  cost_history = zeros(1000,1);
  
  for iter=1:1000
    htheta = sigmoid(x*theta);
    new_theta = zeros(size(theta,1),1);
    for feature=1:size(theta,1)
      new_theta(feature) = theta(feature) - alpha * sum((htheta - y) .*x(:,feature))                         
    end
    theta = new_theta;
    cost_history(iter) = computeCost(x,y,theta);
  end
  end
  </code></pre>