You&#x27;ve &quot;inflicted scale&quot; but with constraints that makes it reasonable to handle:<p>As long as you first write the tweet to a durable canonical datastore, you can recover from the loss of any queue for individual users by re-processing the timelines of the accounts they follow. Hence you don&#x27;t need those queues to be particularly durable. You can afford to cache a lot of data in memory, and write compressed updates to disk lazily. How durable you make the system is a tradeoff between the frequency of node failure and the cost of recovering from it vs. the cost of more nodes to handle lower write rates.<p>The alternative is to pull, but then you inflict polling costs + a lot of caching hassle.