Thanks, that answers my question.<p>Considering more than one exact txn I imagine will hit a single specific node often, at large scale with a single mode down even if that means 5% of transactions block, you are basically growing a queue of waiting work indefinitely with the only upper bound being how much ram you have.  Meanwhile 5% of clients will be waiting and this node may take awhile to come back if it needs something.<p>Once your out of ram &#x2F; etc constraints the 5% of the system that is not functioning turns into 100% real fast because your capacity to handle the other 95% takes ram or other resources you now have dedicated to waiting.<p>If what your saying is also each client can only have one blocked transaction that is relevant but doesn&#x27;t prevent a consumer to spin up as many clients in succession trying to get through.<p>I would suggest that you have at minimum strict timeouts for the the transactions, in conjunction with an immediate fail if the node is answer is not available right now. So a client would never wait more than X seconds or the transaction is aborted and if necessary rolled back.<p>What this would create is a system with predictable failure cases when things go pear shaped. You could calculate in advance how much overhead it would add when something goes wrong as you can have a determinant time factor when having clients wait instead of indefinitely.<p>Furthermore , what if a node never comes back. Somehow there seems to need a transaction failure that is handed back to the client whether it&#x27;s node is down, node is gone forever , or node simply timed out.<p>At the end of the day even if your system is able to handle N thousands of transactions pleasantly waiting and can still answer other requests indefinitely that is a great accomplishment, but in practice may not be ideal for many workloads.  People and computers tend to both retry interactions with data that are slow or failed and the combination of just taking on more and more work and hoping everything flushes out when things become healthy is a recipe for a thundering herd, and better served by something like an async work queue type of pattern.<p>Btw I say this w&#x2F;o looking at your code , just home page, so possible these things exist and it&#x27;s not clear what the bounds and failure cases are yet.<p>Keep on hacking on it!