The Evolution of Container Usage at Netflix
Netflix seems so over engineered to me. They basically have a catalog of a few thousand movies that are negotiated months in advance of actual use. Basically, they just need to encode them and put them on a box and ship them to edge caches. Caching immutable data scales incredibly well.I would also bet that 99.99% of the movies people actually watch on Netflix, would fit on a single box.In regard to the analytics, they have 100 million subscribers. Let's say each subscriber watches an average of 100 episodes/movies a day. For each watch you record subscriber ID, movie ID, start time, stop time and get 32 bytes * 100 * 100,000,000 = 320 gigabytes of data per day total. I am pretty sure that you could get a commercial database and and business intelligence package that could support the type of analytics you need (mainly clustering analysis) at that scale. A national grocery chain, probably has a similar amount of data ingestion and a similar analytics need. In addition, I have subscribed to multiple Netflix type services and I have never weighed the quality of suggestions very high, giving much more weight to the functionality of the client, lack of ads, and large catalog of good movies.As evidence that this is a solved engineering problem, just look at the number of similar movie services: Amazon, Google play, Hulu, PlayStation Vue, Pureflix, Crackle, etc.Google, Facebook, Baidu, Amazon, the self driving car companies are doing cutting edge stuff in terms of scalability and analysis, but not Netflix. The complexity of their operations, seems mainly to be one of their own doing and not intrinsic to the service they provide.So I look at stuff like the article here and see a bunch of very smart engineers who are bored with the (solved) core problem and spend their time making cool stuff which is actually a pretty good thing.