Notes on Distributed Systems for Young Bloods (2013)
I have general and abstract problems for large systems that I haven&#x27;t really figured out easy solutions for other than using metrics and logs:<p>* Downstream dependency failures and isolation (circuit breaker pattern sort of helps but what do you set the thresholds to?)<p>* When to split resources to different machines (ie resource contention) (if you do it you then have more coordination issues)<p>* Alerting intelligently with out causing fatigue.<p>* Reducing latency at scale (this I haven&#x27;t done well but some hit multiple duplicate services at the same time and pick the fastest one ala google).<p>* The index problem... if we add the index we will improve read speed but at the cost of write speed. Some systems have a careful balance or is dynamic depending on environment (time of day).<p>* Queues are easy to understand and implement but can fail in epic catastrophe. The alternatives are often hard to understand and implement (ie the backpressure problem).