Answers.Body: <p>According to <a href="https://github.com/sovrasov/flops-counter.pytorch/issues/16" rel="nofollow noreferrer">this source</a>, one MAC is roughly equal to two FLOP (multiply accumulate). My guess/understanding would be that the distinction is made because neural nets spend compute overwhelmingly on multiply-accumulate operations, and thus optimizations and statistics over MAC operations would be more significant than FLOPs.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-10-04T17:02:52.650Z
Answers.Text: Comments are not for extended discussion; this conversation has been [moved to chat](https://chat.stackexchange.com/rooms/130233/discussion-on-answer-by-k-c-sayz-k-c-sayz-are-mult-adds-and-flops-equivalent).
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-09-09T04:33:12.050Z
Answers.LastActivityDate: 2021-10-04T15:35:46.353Z
Answers.LastEditDate: 2021-10-04T15:35:46.353Z
Answers.UserDisplayName: k.c. sayz 'k.c sayz'
Body: <p>I am comparing different CNN architectures for edge implementation. Some papers describing architectures refer to <em>mult-adds</em>, like <a href="https://arxiv.org/pdf/1704.04861.pdf" rel="noreferrer">the MobileNet V1 paper</a>, where it is claimed that this net has 569M mult-adds, and others refer to floating-point operations (FLOPs), like <a href="https://arxiv.org/pdf/1711.09224.pdf" rel="noreferrer">the CondenseNet paper</a> claims 274M FLOPs.</p>
  <p>Are these comparable? Is 1 multiply-add equivalent to 2 floating-point operations? Any direction will be greatly appreciated.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-09-08T22:23:50.737Z
Comments.Text: One of [the papers that you mention](https://arxiv.org/pdf/1711.09224.pdf) states "_Throughout the paper, FLOPs refers to the number of multiplication-addition operations_", and this is consistent with my knowledge of FLOPS, which are pretty standard in computational science (i.e. analysis and implementation of numerical algorithms, etc.). However, it is the first time I hear of "mult-add", to be honest, so I would need to check what those papers actually refer to when they calculate the "mult-adds".
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-09-08T22:35:33.563Z
Comments.Text: [This answer](https://stackoverflow.com/a/51082939/3924118) says they are equivalent, but it doesn't explain why. However, [this answer](https://stackoverflow.com/a/51866343/3924118) seems to provide more insight. Maybe later I will provide a more formal answer.
Comments.UserDisplayName: nbro
Tags: <convolutional-neural-networks><terminology><papers><complexity-theory>
Title: Are mult-adds and FLOPs equivalent?
UserDisplayName: Quintus
ThreadText: # Are mult-adds and FLOPs equivalent?
  
  (Asked by: username_0 on 2020-09-08)
  
  I am comparing different CNN architectures for edge implementation. Some papers describing architectures refer to *mult-adds*, like [the MobileNet V1 paper](https://arxiv.org/pdf/1704.04861.pdf), where it is claimed that this net has 569M mult-adds, and others refer to floating-point operations (FLOPs), like [the CondenseNet paper](https://arxiv.org/pdf/1711.09224.pdf) claims 274M FLOPs.
  
  Are these comparable? Is 1 multiply-add equivalent to 2 floating-point operations? Any direction will be greatly appreciated.
  
  **Tags:** convolutional-neural-networks, terminology, papers, complexity-theory
  > **Comments:**
  > **username_1**: One of [the papers that you mention](https://arxiv.org/pdf/1711.09224.pdf) states "_Throughout the paper, FLOPs refers to the number of multiplication-addition operations_", and this is consistent with my knowledge of FLOPS, which are pretty standard in computational science (i.e. analysis and implementation of numerical algorithms, etc.). However, it is the first time I hear of "mult-add", to be honest, so I would need to check what those papers actually refer to when they calculate the "mult-adds".  
  > **username_1**: [This answer](https://stackoverflow.com/a/51082939/3924118) says they are equivalent, but it doesn't explain why. However, [this answer](https://stackoverflow.com/a/51866343/3924118) seems to provide more insight. Maybe later I will provide a more formal answer.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2020-09-09. Score: 4
  
  According to [this source](https://github.com/sovrasov/flops-counter.pytorch/issues/16), one MAC is roughly equal to two FLOP (multiply accumulate). My guess/understanding would be that the distinction is made because neural nets spend compute overwhelmingly on multiply-accumulate operations, and thus optimizations and statistics over MAC operations would be more significant than FLOPs.
  
  > **Comments:**
  > **username_1**: Comments are not for extended discussion; this conversation has been [moved to chat](https://chat.stackexchange.com/rooms/130233/discussion-on-answer-by-k-c-sayz-k-c-sayz-are-mult-adds-and-flops-equivalent).  
  
  ---
  
  