An introduction to RabbitMQ
I&#x27;ve had truly terrible experiences with RabbitMQ. I believe that it should not be used in any application where message loss is not acceptable. Its two big problems are that it cannot tolerate network partitions (reason enough to never use it in production systems, see <a href="https:&#x2F;&#x2F;twitter.com&#x2F;antifuchs&#x2F;status&#x2F;735628465924243456" rel="nofollow">https:&#x2F;&#x2F;twitter.com&#x2F;antifuchs&#x2F;status&#x2F;735628465924243456</a>), and it provides no backpressure to producers when it starts running out of memory.<p>In my last job, we used Rabbit to move about 15k messages per sec across about 2000 queues with 200 producers (which produced to all queues) and 2000 consumers (which each read from their own queues). Any time any of the consumers would slow down of fail, rabbit would run out of memory and crash, causing sitewide failure.<p>Additionally, Rabbit would invent network partitions out of thin air, which would cause it to lose messages, as when partitions are healed, all messages on an arbitrarily chosen side of the partition are discarded. (See <a href="https:&#x2F;&#x2F;aphyr.com&#x2F;posts&#x2F;315-jepsen-rabbitmq" rel="nofollow">https:&#x2F;&#x2F;aphyr.com&#x2F;posts&#x2F;315-jepsen-rabbitmq</a> for more details about Rabbit&#x27;s issues and some recommendations for running Rabbit, which sound worse than just using something else to me.)<p>We experimented with &quot;high availability&quot; mode, which caused the cluster to crash more frequently and lose more messages, &quot;durability&quot;, which caused the cluster to crash more frequently and lose more messages, and trying to colocate all of our Rabbit nodes on the same rack (which did not fix the constant partitions, and caused us to totally fail when this rack lost power, as you&#x27;d expect.)<p>These are not theoretical problems. At one point, I spent an entire night fighting with this stupid thing alongside 4 other competent infrastructure engineers. The only long term solution that we found was to completely deprecate our use of Rabbit and use Kafka instead.<p>To anyone considering Rabbit, please reconsider! If you&#x27;re OK with losing messages, then simply making an asynchronous fire-and-forget RPC directly to the relevant consumers may be a better solution for you, since at least there isn&#x27;t more infrastructure to maintain.