Reason #1: Many (most?) computational tasks are <i>not</i> easily parallelizable. The canonical analogy is making a baby:<p>- One woman can make a baby in nine months
- Nine women can make nine babies in nine months
- But nine women can&#x27;t make a baby in one month<p>Each step of fetal development depends on the previous step. For a given single baby, you can&#x27;t have one woman working on step #49 while another woman works on step #17.<p>A lot of computational tasks are similar. Think of the Fibonacci sequence: the computation of each number depends on previous results.<p>Reason #2: The overhead of communication and coordination. If you split up a task amongst 32 cores, those cores need to communicate with the parent process and perhaps with each other as well. This eats into your transistor budget.<p>Reason #3: Resource contention. It&#x27;s nice that you can split up your memory bandwidth-intense task across 128 cores, but if most of those cores are just sitting around and idling because of memory bandwidth contention issues, you haven&#x27;t gained anything.<p>Reason #4: It&#x27;s hard. Programmers struggle to write parallel code. Some tools make it much easier of course.