Answers.Body: <p>Yes, research into ultra-low precision neural network is generally referred to as network quantization. For example, the weights and actications of an artificial neural network can be quantized down to 4-bit, or in extreme cases even 2-bit and 1-bit (binary neural networks).</p>
  <p>This is a good introductory article to start with:
  <a href="https://arxiv.org/abs/2106.08295" rel="nofollow noreferrer">https://arxiv.org/abs/2106.08295</a>, which goes into detail how network quantization can be done to the maximum extent of preserving the original network accuracy.</p>
  <p>The applications of network quantization is immediate. A network with lower precisions would use much less memory and compute power, which is particularly important on edge devices.</p>
  <p>However, I do not know whether there are certain ML tasks that are more amenable to quantizatios then others. Generally speaking, we talk about quantization on a network basis. For example, if you quantization ResNet-50, you can generally use this network to run multiple CV tasks like classification, detection, etc.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-10-15T15:49:31.140Z
Answers.LastActivityDate: 2022-10-15T16:03:20.657Z
Answers.LastEditDate: 2022-10-15T16:03:20.657Z
Answers.UserDisplayName: PeaBrane
Answers.Body: <p>It’s a tradeoff allowing you to fit a larger model into a fixed RAM budget (ie the size of your GPU). Whether this is a good tradeoff is model- and data-specific, but anecdotally I’ve had good luck with it and usually use half precision to good effect (NLP, mostly).</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-01-17T08:20:01.443Z
Answers.Text: Please make a note of the change in the question.
Answers.UserDisplayName: hanugm
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-07-22T05:42:11.367Z
Answers.LastActivityDate: 2021-07-22T05:42:11.367Z
Answers.UserDisplayName: G__
Body: <p>While reading about <a href="https://pytorch.org/docs/stable/generated/torch.nn.Module.html" rel="nofollow noreferrer">Module</a> in PyTorch, I came across a new data type called <code>half</code> datatype.</p>
  <p><code>half()</code> method when calls on a Module casts all floating-point parameters and buffers to <strong>half datatype</strong>.</p>
  <p>It is a 16-bit floating-point number as mentioned <a href="https://en.wikipedia.org/wiki/Half-precision_floating-point_format" rel="nofollow noreferrer">here</a>.</p>
  <p>It is mentioned in Wikipedia that</p>
  <blockquote>
  <p>It is intended for storage of floating-point values in <strong>applications
  where higher precision is not essential</strong> for performing arithmetic
  computations.</p>
  </blockquote>
  <p>It implies that the precision of parameters (say, weights for a neural network) is not important in certain applications and hence one can use half datatype while implementing a neural network.</p>
  <p>Did any research support the statement that <strong>precision, that is the range of values it takes, of weights, is unimportant</strong> for certain applications?</p>
  
Tags: <neural-networks><reference-request><pytorch><implementation><weights>
Title: What are the applications in which the precision of the neural network's weights is unimportant?
UserDisplayName: hanugm
ThreadText: # What are the applications in which the precision of the neural network's weights is unimportant?
  
  (Asked by: username_0 on 2021-07-19)
  
  While reading about [Module](https://pytorch.org/docs/stable/generated/torch.nn.Module.html) in PyTorch, I came across a new data type called `half` datatype.
  
  `half()` method when calls on a Module casts all floating-point parameters and buffers to **half datatype**.
  
  It is a 16-bit floating-point number as mentioned [here](https://en.wikipedia.org/wiki/Half-precision_floating-point_format).
  
  It is mentioned in Wikipedia that
  
  > It is intended for storage of floating-point values in **applications
  > where higher precision is not essential** for performing arithmetic
  > computations.
  
  It implies that the precision of parameters (say, weights for a neural network) is not important in certain applications and hence one can use half datatype while implementing a neural network.
  
  Did any research support the statement that **precision, that is the range of values it takes, of weights, is unimportant** for certain applications?
  
  **Tags:** neural-networks, reference-request, pytorch, implementation, weights
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2022-10-15. Score: 1
  
  Yes, research into ultra-low precision neural network is generally referred to as network quantization. For example, the weights and actications of an artificial neural network can be quantized down to 4-bit, or in extreme cases even 2-bit and 1-bit (binary neural networks).
  
  This is a good introductory article to start with:
  <https://arxiv.org/abs/2106.08295>, which goes into detail how network quantization can be done to the maximum extent of preserving the original network accuracy.
  
  The applications of network quantization is immediate. A network with lower precisions would use much less memory and compute power, which is particularly important on edge devices.
  
  However, I do not know whether there are certain ML tasks that are more amenable to quantizatios then others. Generally speaking, we talk about quantization on a network basis. For example, if you quantization ResNet-50, you can generally use this network to run multiple CV tasks like classification, detection, etc.
  
  
  ---
  
  ### Answer by username_2 on 2021-07-22. Score: 0
  
  It’s a tradeoff allowing you to fit a larger model into a fixed RAM budget (ie the size of your GPU). Whether this is a good tradeoff is model- and data-specific, but anecdotally I’ve had good luck with it and usually use half precision to good effect (NLP, mostly).
  
  > **Comments:**
  > **username_0**: Please make a note of the change in the question.  
  
  ---
  
  