Answers.Body: <p>So generally, when you seperate your training data to 80%-20% then you fit method should get <strong>2</strong> <code>x</code>,<code>y</code>. better to call them <code>x_train</code>,<code>y_train</code>, <code>x_val</code>, <code>y_val</code> or something similar.</p>
  <p>Now its important you do the split before entering the fit, and not do it for each epoch or something alike.</p>
  <p>Once you do that and the fit method should be something like:</p>
  <pre><code>def fit(self, x_train, y_train, x_val, y_val, num_iter=10, mb_size=32, lr=1e-1, lambda_reg=1e-2, reset=True):
  </code></pre>
  <p>Then you should, at the end of each epoch, test the performance of the model on the validation set entirely and calculate the desired metric for evaluation. If you improved it's better to save the current model. This is done repeatedly for each epoch until the end of the training and you will guarantee to have the model who gave you the best results on the validation set rather than on the training set, which might be overfitting it.</p>
  <p>I will do it in a separate method with the following flow:</p>
  <ol>
  <li>iterate on each sample in the validation set</li>
  <li>for each one calculate the loss/metric</li>
  <li>append it to some list</li>
  <li>return the mean of that list</li>
  </ol>
  <p>And if the average result is better from some previously saved one, save the new model</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-10-01T07:26:57.140Z
Answers.Text: Thank you for your answer.
  What is the purpose of writing x_val and y_val inside fit?
  In the separate method can I pass the x_val and y_val and then compute the loss?
Answers.UserDisplayName: DeepakS
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-10-01T07:30:34.987Z
Answers.Text: @DeepakSonker you need to do the separation beforehand, and at the moment you have x,y data but there is no distinction between the training and validation
Answers.UserDisplayName: David
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-10-01T07:39:10.623Z
Answers.Text: How to use the best loss to test the test_data? how will I implement that loss in my code? I am really confused. I have not done this previously.
Answers.UserDisplayName: DeepakS
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-10-01T08:04:45.650Z
Answers.Text: @DeepakSonker So you will save the current best loss/acc and for each time you evaluate the results on the validation set you will examine the new results. if its better save the model if not skip; or something similar
Answers.UserDisplayName: David
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-10-01T08:22:00.553Z
Answers.Text: Can you provide me the reading material where I can find the implementation of this?
Answers.UserDisplayName: DeepakS
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-10-01T05:36:29.787Z
Answers.LastActivityDate: 2020-10-01T05:36:29.787Z
Answers.UserDisplayName: David
Body: <p>I am new to machine learning and recently I joined a course where I was given a logistic regression assignment in which I had to split 20% of the training dataset for the validation dataset and then use the validation dataset to capture the minimum possible loss and then use the test dataset to find the accuracy of the model.
  Below is my code for implementing logistic regression</p>
  <pre><code>class LogReg(LinReg):
      def __init__(self, n_dim, bias=True):
          if bias:
              n_dim = n_dim + 1
          super(LogReg, self).__init__(n_dim)
          self.bias = bias
      
    def __call__(self, x):
        return x.mm(self.theta).sigmoid()
  
    def compute_loss(self, x, y, lambda_reg):
        # The function has a generic implementation, and can also work for the neural nets!
        predictions = self(x)
        loss = -(y * torch.log(predictions) + (1-y) * torch.log(1 - predictions)).mean()
        regularizer = self.theta.transpose(0, 1).mm(self.theta)
        return loss + regularizer.mul(lambda_reg)
    @staticmethod
    def add_bias(x):
        ones = torch.ones((x.size(0), 1), dtype=torch.float32)
        x_hat = torch.cat((ones, x), dim=-1)
        return x_hat
  
    def fit(self, x, y, num_iter=10, mb_size=32, lr=1e-1, lambda_reg=1e-2, reset=True):
        N = x.size(0)
        losses = []
        x_hat = x
        # Adding a bias term if needed
        if self.bias:
            x_hat = self.add_bias(x)
        if reset:
            self.reset() # Very important if you want to call fit multiple times
        num_batches = x.size(0) // mb_size
        # The outer loop goes over `epochs`
        # The inner loop goes over the whole training data
        for it in range(num_iter):
            loss_per_epoch = 0
            for batch_it in range(num_batches):
                # has been implemented for the linear model
                self.zero_grad()
  
                ind = torch.randint(0, N, (mb_size, 1)).squeeze()
                x_mb, y_mb = x_hat[ind, :], y[ind, :]
  
                loss = self.compute_loss(x_mb, y_mb, lambda_reg)
  
                loss.backward()
                self.theta.data = self.theta.data - lr*self.grad().data
                loss_per_epoch += loss.item()
            
            loss_per_epoch /= num_batches
            losses.append(loss_per_epoch)
        
        return losses
  </code></pre>
  <p>How should I use the validation set at the level of epoch to find the best loss?</p>
  
Tags: <python><pytorch><regression>
Title: How to use validation dataset in my logistic regression model?
UserDisplayName: DeepakS
ThreadText: # How to use validation dataset in my logistic regression model?
  
  (Asked by: username_0 on 2020-10-01)
  
  I am new to machine learning and recently I joined a course where I was given a logistic regression assignment in which I had to split 20% of the training dataset for the validation dataset and then use the validation dataset to capture the minimum possible loss and then use the test dataset to find the accuracy of the model.
  Below is my code for implementing logistic regression
  
  ```
  class LogReg(LinReg):
      def __init__(self, n_dim, bias=True):
          if bias:
              n_dim = n_dim + 1
          super(LogReg, self).__init__(n_dim)
          self.bias = bias
      
    def __call__(self, x):
        return x.mm(self.theta).sigmoid()
  
    def compute_loss(self, x, y, lambda_reg):
        # The function has a generic implementation, and can also work for the neural nets!
        predictions = self(x)
        loss = -(y * torch.log(predictions) + (1-y) * torch.log(1 - predictions)).mean()
        regularizer = self.theta.transpose(0, 1).mm(self.theta)
        return loss + regularizer.mul(lambda_reg)
    @staticmethod
    def add_bias(x):
        ones = torch.ones((x.size(0), 1), dtype=torch.float32)
        x_hat = torch.cat((ones, x), dim=-1)
        return x_hat
  
    def fit(self, x, y, num_iter=10, mb_size=32, lr=1e-1, lambda_reg=1e-2, reset=True):
        N = x.size(0)
        losses = []
        x_hat = x
        # Adding a bias term if needed
        if self.bias:
            x_hat = self.add_bias(x)
        if reset:
            self.reset() # Very important if you want to call fit multiple times
        num_batches = x.size(0) // mb_size
        # The outer loop goes over `epochs`
        # The inner loop goes over the whole training data
        for it in range(num_iter):
            loss_per_epoch = 0
            for batch_it in range(num_batches):
                # has been implemented for the linear model
                self.zero_grad()
  
                ind = torch.randint(0, N, (mb_size, 1)).squeeze()
                x_mb, y_mb = x_hat[ind, :], y[ind, :]
  
                loss = self.compute_loss(x_mb, y_mb, lambda_reg)
  
                loss.backward()
                self.theta.data = self.theta.data - lr*self.grad().data
                loss_per_epoch += loss.item()
            
            loss_per_epoch /= num_batches
            losses.append(loss_per_epoch)
        
        return losses
  
  ```
  
  How should I use the validation set at the level of epoch to find the best loss?
  
  **Tags:** python, pytorch, regression
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-10-01. Score: 1 | Accepted answer
  
  So generally, when you seperate your training data to 80%-20% then you fit method should get **2** `x`,`y`. better to call them `x_train`,`y_train`, `x_val`, `y_val` or something similar.
  
  Now its important you do the split before entering the fit, and not do it for each epoch or something alike.
  
  Once you do that and the fit method should be something like:
  
  ```
  def fit(self, x_train, y_train, x_val, y_val, num_iter=10, mb_size=32, lr=1e-1, lambda_reg=1e-2, reset=True):
  
  ```
  
  Then you should, at the end of each epoch, test the performance of the model on the validation set entirely and calculate the desired metric for evaluation. If you improved it's better to save the current model. This is done repeatedly for each epoch until the end of the training and you will guarantee to have the model who gave you the best results on the validation set rather than on the training set, which might be overfitting it.
  
  I will do it in a separate method with the following flow:
  
  1. iterate on each sample in the validation set
  2. for each one calculate the loss/metric
  3. append it to some list
  4. return the mean of that list
  
  And if the average result is better from some previously saved one, save the new model
  
  > **Comments:**
  > **username_0**: Thank you for your answer. What is the purpose of writing x_val and y_val inside fit? In the separate method can I pass the x_val and y_val and then compute the loss?  
  > **username_1**: @DeepakSonker you need to do the separation beforehand, and at the moment you have x,y data but there is no distinction between the training and validation  
  > **username_0**: How to use the best loss to test the test_data? how will I implement that loss in my code? I am really confused. I have not done this previously.  
  > **username_1**: @DeepakSonker So you will save the current best loss/acc and for each time you evaluate the results on the validation set you will examine the new results. if its better save the model if not skip; or something similar  
  > **username_0**: Can you provide me the reading material where I can find the implementation of this?  
  
  ---
  
  