Answers.Body: <p>The size of the parameters tensor is depended on what type of layer that you want to build. Convolutional, fully connected, attention or even custom layer, each layer has a difference in the way it treats input, reading the documents is the good way to start (<a href="https://cs231n.github.io/convolutional-networks/#fc" rel="nofollow noreferrer">CS231n of Stanford University</a> describes in detail each layer's properties).</p>
  <p>In your case, the layer name is the fully-connected layer (or dense, linear layer in other documents) which converts an m-dimension input vector to an n-dimension output vector. All m nodes are mapped to n nodes by a <span class="math-container">$m\times n$</span> matrix (that's why its name is fully-connected) and the bias vector, in short, is to help the learned function be more flexible.</p>
  <p>Therefore, the rule to decide the size of weight and bias is the size of input and target vector. Below is a simple example that builds a Linear layer from scratch:</p>
  <pre><code>import torch
  class Example_Linear(torch.nn.Module):
      '''
      An example of Linear layer:
      Args:
          :param in_d: number of input dimensions
          :param out_d: number of output dimensions
          :param bias: whether to use bias with the linear layer
      '''
      def __init__(self, in_d, out_d, bias=True):
          super().__init__()
          self.weight = torch.nn.Parameter(torch.randn(out_d, in_d))
          self.bias = None
          if bias:
              self.bias = torch.nn.Parameter(torch.FloatTensor(out_d).fill_(0))
          
      def forward(self, x):
          out = torch.matmul(x,self.weight.T)
          out += self.bias
          return out
  
  in_dimension = 10
  out_dimension = 1
  model = Example_Linear(in_d=in_dimension, out_d=out_dimension, bias=True)
  </code></pre>
  <p>Or you can simply implement fully-connected layer by one line:</p>
  <pre><code>model = torch.nn.Linear(in_dimension, out_dimension, bias=True)
  </code></pre>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-12-21T13:40:45.333Z
Answers.Text: Thanks CucaRot, very helpful and clears my queries.. My first tensors take 3 inputs and sencond tensor is 2 targets (output)
Answers.UserDisplayName: ZKS
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-12-23T08:00:33.237Z
Answers.Text: Good to hear that
Answers.UserDisplayName: CuCaRot
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-12-21T09:42:52.820Z
Answers.LastActivityDate: 2021-12-21T09:42:52.820Z
Answers.UserDisplayName: CuCaRot
Body: <p>Below are the two tensors</p>
  <pre><code>[ 73.,  67.,  43.]    
  [ 91.,  88.,  64.],
  [ 87., 134.,  58.],
  [102.,  43.,  37.],
  [ 69.,  96.,  70.]
  
  [ 56.,  70.],
  [ 81., 101.],
  [119., 133.],
  [ 22.,  37.],
  [103., 119.]
  </code></pre>
  <p>These are the weight that are added</p>
  <h1>Weights and biases</h1>
  <pre><code> w = torch.randn(2, 3, requires_grad=True)
   b = torch.randn(2, requires_grad=True)
  </code></pre>
  <p>I am not able to understand how the size of tensors are decided for weight and biases. Is there common rule that we should follow while adding weight and biases for our model</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-12-21T22:14:31.407Z
Comments.Text: Is this is a programming issue/question?
Comments.UserDisplayName: nbro
Tags: <pytorch><weights><linear-regression><tensor><bias>
Title: Not able to understand Pytorch Tensor (Weight & Biases) Size for Linear Regression
UserDisplayName: ZKS
ThreadText: # Not able to understand Pytorch Tensor (Weight & Biases) Size for Linear Regression
  
  (Asked by: username_0 on 2021-12-21)
  
  Below are the two tensors
  [ 73.,  67.,  43.]    
  [ 91.,  88.,  64.],
  [ 87., 134.,  58.],
  [102.,  43.,  37.],
  [ 69.,  96.,  70.]
  
  [ 56.,  70.],
  [ 81., 101.],
  [119., 133.],
  [ 22.,  37.],
  [103., 119.]
  
  These are the weight that are added
  Weights and biases
   w = torch.randn(2, 3, requires_grad=True)
   b = torch.randn(2, requires_grad=True)
  
  I am not able to understand how the size of tensors are decided for weight and biases. Is there common rule that we should follow while adding weight and biases for our model
  
  **Tags:** pytorch, weights, linear-regression, tensor, bias
  > **Comments:**
  > **username_1**: Is this is a programming issue/question?  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2021-12-21. Score: 3 | Accepted answer
  
  The size of the parameters tensor is depended on what type of layer that you want to build. Convolutional, fully connected, attention or even custom layer, each layer has a difference in the way it treats input, reading the documents is the good way to start (CS231n of Stanford University describes in detail each layer's properties).
  In your case, the layer name is the fully-connected layer (or dense, linear layer in other documents) which converts an m-dimension input vector to an n-dimension output vector. All m nodes are mapped to n nodes by a $m\times n$ matrix (that's why its name is fully-connected) and the bias vector, in short, is to help the learned function be more flexible.
  Therefore, the rule to decide the size of weight and bias is the size of input and target vector. Below is a simple example that builds a Linear layer from scratch:
  import torch
  class Example_Linear(torch.nn.Module):
      '''
      An example of Linear layer:
      Args:
          :param in_d: number of input dimensions
          :param out_d: number of output dimensions
          :param bias: whether to use bias with the linear layer
      '''
      def __init__(self, in_d, out_d, bias=True):
          super().__init__()
          self.weight = torch.nn.Parameter(torch.randn(out_d, in_d))
          self.bias = None
          if bias:
              self.bias = torch.nn.Parameter(torch.FloatTensor(out_d).fill_(0))
          
      def forward(self, x):
          out = torch.matmul(x,self.weight.T)
          out += self.bias
          return out
  
  in_dimension = 10
  out_dimension = 1
  model = Example_Linear(in_d=in_dimension, out_d=out_dimension, bias=True)
  
  Or you can simply implement fully-connected layer by one line:
  model = torch.nn.Linear(in_dimension, out_dimension, bias=True)
  
  > **Comments:**
  > **username_0**: Thanks CucaRot, very helpful and clears my queries.. My first tensors take 3 inputs and sencond tensor is 2 targets (output)  
  > **username_2**: Good to hear that  
  
  ---
  
  