This article is a chapter from my new book Explain the Cloud Like I'm 10. The first release was written specifically for cloud newbies. I've made some updates and added a few chapters—Netflix: What Happens When You Press Play? and What is Cloud Computing?—that level it up to a couple ticks past beginner. I think even fairly experienced people might get something out of it.
I've also created a somewhat expanded version of the article in a standalone Kindle ebook. You can find the ebook at Netflix: What Happens When You Press Play?
So if you are looking for a good introduction to the cloud or know someone who is, please take a look. I think you'll like it. I'm pretty proud of how it turned out.
Netflix seems so simple. Press play and video magically appears. Easy, right? Not so much.
Given our discussion in the What is Cloud Computing? chapter, you might expect Netflix to serve video using AWS. Press play in a Netflix application and video stored in S3 would be streamed from S3, over the internet, directly to your device.
A completely sensible approach…for a much smaller service.
But that's not how Netflix works at all. It's far more complicated and interesting than you might imagine.
To see why let's look at some impressive Netflix statistics for 2017.
Netflix has more than 110 million subscribers.
Netflix operates in more than 200 countries.
Netflix has nearly $3 billion in revenue per quarter.
Netflix adds more than 5 million new subscribers per quarter.
Netflix plays more than 1 billion hours of video each week. As a comparison, YouTube streams 1 billion hours of video every day while Facebook streams 110 million hours of video every day.
Netflix played 250 million hours of video on a single day in 2017.
Netflix accounts for over 37% of peak internet traffic in the United States.
Netflix plans to spend $7 billion on new content in 2018.
Netflix is huge. They're global, they have a lot of members, they play a lot of videos, and they have a lot of money.
Another relevant factoid is Netflix is subscription based. Members pay Netflix monthly and can cancel at any time. When you press play to chill on Netflix, it had better work. Unhappy members unsubscribe.
Netflix is a terrific example of all the ideas we've talked about, which is why this chapter goes into a lot more detail than the other cloud services we've covered.
One big reason for diving deeper into Netflix is they make much more information available than other companies.
Netflix holds communication as a central cultural value. Netflix more than lives up to its standards.
In fact, I'd like to thank Netflix for being so open about their architecture. Over the years, Netflix has given hundreds of talks and written hundreds of articles on the inner-workings of how they operate. The whole industry is better for it.
Another reason for going into so much detail on Netflix is Netflix is just plain fascinating. Most of us have used Netflix at one time or another. Who wouldn't love peeking behind the curtain to see what makes Netflix tick?
Netflix operates in two clouds: AWS and Open Connect.
How does Netflix keep their members happy? With the cloud of course. Actually, Netflix uses two different clouds: AWS and Open Connect.
Both clouds must work together seamlessly to deliver endless hours of customer-pleasing video.
The three parts of Netflix: client, backend, content delivery network (CDN).
You can think of Netflix as being divided into three parts: the client, the backend, and the content delivery network (CDN).
The client is the user interface on any device used to browse and play Netflix videos. It could be an app on your iPhone, a website on your desktop computer, or even an app on your Smart TV. Netflix controls each and every client for each and every device.
Everything that happens before you hit play happens in the backend, which runs in AWS. That includes things like preparing all new incoming video and handling requests from all apps, websites, TVs, and other devices.
Everything that happens after you hit play is handled by Open Connect. Open Connect is Netflix's custom global content delivery network (CDN). Open Connect stores Netflix video in different locations throughout the world. When you press play the video streams from Open Connect, into your device, and is displayed by the client. Don't worry; we'll talk more about what a CDN is a little later.
Interestingly, at Netflix they don't actually say hit play on video, they say clicking start on a title. Every industry has its own lingo.
By controlling all three areas—client, backend, CDN— Netflix has achieved complete vertical integration.
Netflix controls your video viewing experience from beginning to end. That's why it just works when you click play from anywhere in the world. You reliably get the content you want to watch when you want to watch it.
Let's see how Netflix makes that happen.
Netflix launched in 1998. At first they rented DVDs through the US Postal Service. But Netflix saw the future was on-demand streaming video.
In 2007 Netflix introduced their streaming video-on-demand service that allowed subscribers to stream television series and films via the Netflix website on personal computers, or the Netflix software on a variety of supported platforms, including smartphones and tablets, digital media players, video game consoles, and smart TVs.
On a personal note, that streaming video-on-demand was the future might seem obvious. And it was. I worked at a couple of startups that tried to make a video-on-demand product. They failed.
Netflix succeeded. Netflix certainly executed well, but they were late to the game, and that helped them. By 2007 the internet was fast enough and cheap enough to support streaming video services. That was never the case before. The addition of fast, low-cost mobile bandwidth and the introduction of powerful mobile devices like smart phones and tablets, has made it easier and cheaper for anyone to stream video at any time from anywhere. Timing is everything.
EC2 was just getting started in 2007, about the same time Netflix's streaming service started. There was no way Netflix could have launched using EC2.
Netflix built two datacenters, located right next to each other. They experienced all the problems we talked about in earlier chapters.
Building out a datacenter is a lot of work. Ordering equipment takes a long time. Installing and getting all the equipment working takes a long time. And as soon they got everything working they would run out of capacity, and the whole process had to start over again.
The long lead times for equipment forced Netflix to adopt what is known as a vertical scaling strategy. Netflix made big programs that ran on big computers. This approach is called building a monolith. One program did everything.
The problem is when you're growing really fast like Netflix; it's very hard to make a monolith reliable. And it wasn't.
For three days in August 2008, Netflix could not ship DVDs because of corruption in their database. This was unacceptable. Netflix had to do something.
The experience of building datacenters had taught Netflix an important lesson—they weren't good at building datacenters.
What Netflix was good at was delivering video to their members. Netflix would rather concentrate on getting better at delivering video rather than getting better at building datacenters. Building datacenters was not a competitive advantage for Netflix, delivering video is.
At that time, Netflix decided to move to AWS. AWS was just getting established, so selecting AWS was a bold move.
Netflix moved to AWS because it wanted a more reliable infrastructure. Netflix wanted to remove any single point of failure from its system. AWS offered highly reliable databases, storage and redundant datacenters. Netflix wanted cloud computing, so it wouldn't have to build big unreliable monoliths anymore. Netflix wanted to become a global service without building its own datacenters. None of these capabilities were available in its old datacenters and never would be.
A reason Netflix gave for choosing AWS was it didn't want to do any undifferentiated heavy lifting. Undifferentiated heavy lifting are those things that have to be done, but don't provide any advantage to the core business of providing a quality video watching experience. AWS does all the undifferentiated heavy lifting for Netflix. This lets Netflixians focus on providing business value.
It took more than eight years for Netflix to complete the process of moving from their own datacenters to AWS. During that period Netflix grew its number of streaming customers eightfold. Netflix now runs on several hundred thousand EC2 instances.
It's not like Netflix has never experienced down time on AWS, but on the whole, its service is much more reliable than it was before.
Netflix is so reliable now because they've taken extraordinary steps to make their service reliable.
Netflix operates out of three AWS regions: one in North Virginia, one in Portland Oregon, and one in Dublin Ireland. Within each region, Netflix operates in three different availability zones.
Netflix has said there are no plans to operate out of more regions. It's very expensive and complicated to add new regions. Most companies operate out of just one region, let alone two or three.
The advantage of having three regions is that any one region can fail, and the other regions will step in handle all the members in the failed region. When a region fails, Netflix calls this evacuating a region.
Let's use an example. Let's say you're watching a new House of Cards episode in London England. Because it's closest to London, chances are your Netflix device is connected to the Dublin region.
What happens if the entire Dublin region fails? Does that mean Netflix should stop working for you? Of course not!
Netflix, after detecting the failure, redirects you to Virginia. Your device would now talk to the Virginia region instead of Dublin. You might not even notice there was a failure.
How often does an AWS region fail? Once a month. Well, a region doesn't actually fail every month. Netflix runs monthly tests. Every month Netflix causes a region to fail on purpose just to make sure its system can handle region level failures. A region can be evacuated in six minutes.
Netflix calls this their global services model. Any customer can be served out of any region. This is amazing. And it doesn't happen automatically. AWS has no magic sauce for handling region failures or serving customers out of multiple regions. Netflix has done all this work on its own. Netflix is a pioneer in figuring out how to create reliable systems using multiple regions. I'm not aware of any other company that goes to these lengths to make their service so reliable.
Another advantage of being in these three regions is that it gives Netflix world-wide coverage. Netflix ran some tests and found if you use a Netflix application anywhere in the world, you'll get fast service from one of these three regions.
This may surprise a lot of people, but AWS is cheaper for Netflix. The cloud costs per streaming view ended up being a fraction of the cost of its old datacenters.
Why? The elasticity of the cloud.
Netflix could add servers when it needed them and return them when it didn't. Rather than have a lot of extra computers hanging around doing nothing just to handle peak load, Netflix only had to pay for what was needed, when it was needed.
All the stuff we talked about in the What is Cloud Computing? chapter.
What Happens in AWS Before you Press Play?
Anything that doesn't involve serving video is handled in AWS.
This includes scalable computing, scalable storage, business logic, scalable distributed databases, big data processing and analytics, recommendations, transcoding, and hundreds of other functions.
Don't worry, you don't need to understand what all those things are, but since you may find it interesting, I'll explain them briefly.
Scalable computing and scalable storage.
Scalable computing is EC2 and scalable storage is S3. Nothing new for us here.
Your Netflix device—iPhone, TV, Xbox, Android phone, tablet, etc.—talks to a Netflix service running in EC2.
View a list of potential videos to watch? That's your Netflix device contacting a computer in EC2 to get the list.
Ask for more details about a video? That's your Netflix device contacting a computer in EC2 to get the details.
It's just like all the other cloud services we've talked about in the book.
Netflix uses both DynamoDB and Cassandra for their distributed databases. Not that these names should mean anything to you, they're just high-quality database products.
Database. A database stores data. Your profile information, your billing information, all the movies you've ever watched, all that kind of information is stored in a database.
Distributed. Distributed means the database doesn't run on one big computer, it runs on many computers. Your data is copied to multiple computers so if one or even two computers holding your data fail, your data will be safe. In fact, your data is copied to all three regions. That way, if a region fails your data will be there when the new region is ready to start using it.
Scalable. Scalable means the database can handle as much data as you ever want to put into it. That's one major advantage of being a distributed database. More computers can be added as necessary to handle more data.
Big data processing and analytics.
Big data simply means there's a lot of data. Netflix collects a lot of information. Netflix knows what everyone has watched when they watched it and where they were when they watched. Netflix knows which videos members have looked at but decided not to watch. Netflix knows how many times each video has been watched…and a lot more.
Putting all the data in a standard format is called processing.
Making sense of all that data is called analytics. Data is analyzed to answer specific questions.
Netflix personalizes artwork just for you.
Here's a great example of how Netflix entices you to watch more videos using its data analytics capabilities.
When browsing around looking for something to watch on Netflix, have you noticed there's always an image displayed for each video? That's called the header image.
The header image is meant to intrigue you, to draw you into selecting a video. The idea is the more compelling the header image, the more likely you are to watch a video. And the more videos you watch, the less likely you are to unsubscribe from Netflix.
You might be surprised to learn the image shown for each video is selected specifically for you. Not everyone sees the same image.
Everyone used to see the same header image. Here's how it worked. Members were shown at a random one picture from a group of options, like the pictures in the above Stranger Things collage. Netflix counted every time the video was watched, recording which picture was displayed when the video was selected.
For our Stranger Things example, let's say when the group picture in the center was shown, Stranger Things was watched 1,000 times. For all the other pictures, it was watched only once each.
Since the group picture was the best at getting members to watch, Netflix would make it the header image for Stranger Things forever.
This is called being data-driven. Netflix is known for being a data-driven company. Data is gathered—in this case, the number of views associated with each picture—and used to make the best decisions possible—in this case, which header image to select.
Clever, but can you imagine doing better? Yes, by using more data. That's the theme of the future—solving problems by learning from data.
You and I are likely very different people. Do you think we are motivated by the same kind of header image? Probably not. We have different tastes. We have different preferences.
Netflix knows this too. That's why Netflix now personalizes all the images they show you. Netflix tries to select the artwork highlighting the most relevant aspect of a video to you. How do they do that?
Remember, Netflix records and counts everything you do on their site. They know which kind of movies you like best, which actors you like the most, and so on.
Let's say one of your recommendations is the movie Good Will Hunting. Netflix must choose a header image to show you. The goal is to show an image that lets you know about a movie you'll probably be interested in. Which image should Netflix show you?
If you like comedies, Netflix will show you an image featuring Robin Williams. If you prefer romantic movies, Netflix will show you an image Matt Damon and Minnie Driver poised for a kiss.
By showing Robin Williams, Netflix is letting you know there's likely to be humor in the movie and because Netflix knows you like comedies, this video is a good match.
The Matt Damon and Minnie Driver image conveys a completely different message. If you're a comedy fan and saw this image, you might skip right on by.
That's why selecting the right header image is so important. It sends a strong personalized signal indicating what a movie is about.
Here's another example, Pulp Fiction.
If you've watched a lot of movies starring Uma Thurman, then you're likely to see the header image featuring Uma. If you've watched a lot of movies starring John Travolta, then you're likely to see the header image featuring John.
Can you see how choosing the best possible personalized artwork might make you more likely to watch a particular video?
Netflix appeals to your interests when selecting artwork, yet Netflix doesn't want to lie to you either. They don't want to show a clickbait image just to get you to watch a video you may not like. There's no incentive in that. Netflix isn't paid per video watched. Netflix tries to minimize regret. Netflix wants you to be happy with the videos you watch, so they pick the best header images they can for you.
This is just one small example of how data analysis is used by Netflix. Netflix uses these kind of strategies everywhere.
Usually Netflix will show you only 40 to 50 video options, yet they have many thousands of videos available.
How does Netflix decide? Using machine learning.
That's part of the big data processing and analytics we just talked about. Netflix looks at its data and predicts what you'll like. In fact, everything you see see on a Netflix screen was chosen specifically for you using machine learning.
Here's where we start transitioning into how video is handled by Netflix.
Before you can watch a video on your favorite device of choice, Netflix must convert the video into a format that works best for your device. This process is called transcoding or encoding.
Transcoding is the process that converts a video file from one format to another, to make videos viewable across different platforms and devices.
Netflix encodes all its video in AWS on as many as 300,000 CPUs at one time. That's larger than most super computers!
The source of source media.
Who sends video to Netflix? Production houses and studios. Netflix calls this video source media. The new video is given to the Content Operations Team for processing.
The video comes in a high definition format that's many terabytes in size. A terabyte is big. Imagine 60 stacks of paper as tall as the Eiffel Tower. That's a terabyte.
Before you can view a video, Netflix puts it through a rigorous multi-step process.
The first thing Netflix does is spend a lot of time validating the video. It looks for digital artifacts, color changes, or missing frames that may have been caused by previous transcoding attempts or data transmission problems.
The video is rejected if any problems are found.
After the video is validated, it's fed into what Netflix calls the media pipeline.
A pipeline is simply a series of steps data is put through to make it ready for use, much like an assembly line in a factory. More than 70 different pieces of software have a hand in creating every video.
It's not practical to process a single multi-terabyte sized file, so the first step of the pipeline is to break the video into lots of smaller chunks.
The video chunks are then put through the pipeline so they can be encoded in parallel. In parallel simply means the chunks are processed at the same time.
Let's illustrate parallelism with an example.
Let's say you have one hundred dirty dogs that need washing. Which would be faster, one person washing the dogs one after another? Or would it be faster to hire one hundred dog washers and wash them all the same time?
Obviously, it's faster to have one hundred dog washers working at the same time. That's parallelism. And that's why Netflix uses so many servers in EC2. They need a lot of servers to process these huge video files in parallel. It works too. Netflix says a source media file can be encoded and pushed to their CDN in as little as 30 minutes.
Once the chunks are encoded, they're validated to make sure no new problems have been introduced.
Then the chunks are assembled back into a file and validated once again.
The result is a pile of files.
The encoding process creates a lot of files. Why? The end goal for Netflix is to support every internet-connected device.
Netflix started streaming video in 2007 on Microsoft Windows. Over time more devices were added—Roku, LG, Samsung Blu-ray, Apple Mac, Xbox 360, LG DTV, Sony PS3, Nintendo Wii, Apple iPad, Apple iPhone, Apple TV, Android, Kindle Fire, and Comcast X1.
In all, Netflix supports 2200 different devices. Each device has a video format that looks best on that particular device. If you're watching Netflix on an iPhone, you'll see a video that gives you the best viewing experience on the iPhone.
Netflix calls all the different formats for a video its encoding profile.
Netflix also creates files optimized for different network speeds. If you're watching on a fast network, you'll see higher quality video than you would if you're watching over a slow network.
There are also files for different audio formats. Audio is encoded into different levels of quality and in different languages.
There are also files included for subtitles. A video may have subtitles in a number of different languages.
There are a lot of different viewing options for every video. What you see depends on your device, your network quality, your Netflix plan, and your language choice.
Just how many files are we talking about?
For The Crown, Netflix stores around 1,200 files!
Stranger Things season 2 has even more files. It was shot in 8K and has nine episodes. The source video files were many, many terabytes of data. It took 190,000 CPU hours to encode just one season.
The result? 9,570 different video, audio, and text files!
Let's see how Netflix plays all that video.
Netflix has tried three different video streaming strategies its own small CDN; third-party CDNs; and Open Connect.
Let's start by defining CDN. A CDN is a content distribution network.
Content for Netflix—is of course—the video files we discussed in the previous section.
Distribution means video files are copied from a central location, over a network and stored on computers located all over the world.
For Netflix, the central location where videos are stored is S3.
The idea behind a CDN is simple: put video as close as possible to users by spreading computers throughout the world. When a user wants to watch a video, find the nearest computer with the video on it and stream to the device from there.
The biggest benefits of a CDN are speed and reliability.
Imagine you're watching a video in London and the video is being streamed from Portland, Oregon. The video stream must pass through a lot of networks, including an undersea cable, so the connection will be slow and unreliable.
By moving video content as close as possible to the people watching it, the viewing experience will be as fast and reliable as possible.
Each location with a computer storing video content is called a PoP or point of presence. Each PoP is a physical location that provides access to the internet. It houses servers, routers, and other telecommunications equipment. We'll talk more about PoPs later.
In 2007, when Netflix debuted its new streaming service, it had 36 million members in 50 countries, watching more than a billion hours of video each month, streaming multiple terabits of content per second.
To support the streaming service, Netflix built its own simple CDN in five different locations within the United States.
The Netflix video catalog was small enough at the time that each location contained all of its content.
In 2009, Netflix decided to use 3rd-party CDNs. Around this time, the pricing for 3rd-party CDNs was coming down.
Using 3rd-party CDNs made perfect sense for Netflix. Why spend all the time and effort building a CDN of your own when you can instantly reach the globe using existing CDN services?
Netflix contracted with companies like Akamai, Limelight, and Level 3 to provide CDN services. There's nothing wrong with using third-party CDNs. In fact, pretty much every company does. For example, the NFL has used Akamai to stream live football games.
By not building out its own CDN, Netflix had more time to work on other higher priority projects.
Netflix put a lot of time and effort into developing smarter clients. Netflix created algorithms to adapt to changing networks conditions. Even in the face of errors, overloaded networks, and overloaded servers, Netflix wants members always viewing the best picture possible. One technique Netflix developed is switching to a different video source—say another CDN, or a different server—to get a better result.
At the same time, Netflix was also devoting a lot of effort into all the AWS services we talked about earlier. Netflix calls the services in AWS its control plane. Control plane is a telecommunications term identifying the part of the system that controls everything else. In your body, your brain is the control plane; it controls everything else.
Then Netflix thought it could do better by developing it's own CDN.
In 2011, Netflix realized at its scale it needed a dedicated CDN solution to maximize network efficiency. Video distribution is a core competency for Netflix and could be a huge competitive advantage.
So Netflix started developing Open Connect, its own purpose-built CDN. Open Connect launched in 2012.
Less expensive. 3rd-party CDNs are expensive. Doing it themselves would save a lot of money.
Better quality. By controlling the entire video path—transcoding, CDN, clients on devices—Netflix reasoned it could deliver a superior video viewing experience.
More Scalable. Netflix has the goal of providing service everywhere in the world. Quickly supporting all those people while providing a quality video viewing experience required building its own system.
The 3rd-party CDNs must support users accessing any kind of content from anywhere in the world. Netflix has a much simpler job.
Netflix knows exactly who its users are because they must subscribe to Netflix. Netflix knows exactly which videos it needs to serve. Just knowing it only has to serve large video streams allows Netflix to make a lot of smart optimization choices other CDNs can't make. Netflix also knows a lot about it members. The company knows which videos they like to watch and when they like to watch them.
With this kind of knowledge, Netflix built a really high-performing CDN. Let's go into more details on how Open Connect works.
Remember how we said a CDN has computers distributed all over the world?
Netflix developed its own computer system for video storage. Netflix calls them Open Connect Appliances or OCAs.
There are many OCAs in the above picture. OCAs are grouped into clusters of multiple servers.
Each OCA is a fast server, highly optimized for delivering large files, with lots and lots of hard disks or flash drives for storing video.
There are several different kinds of OCAs for different purposes. There are large OCAs that can store Netflix's entire video catalog. There are smaller OCAs that can store only a portion of Netflix's video catalog. Smaller OCAs are filled with video every day, during off-peak hours, using a process Netflix calls proactive caching. We'll talk more about how proactive caching works later.
From a hardware perspective, there's nothing special about OCAs. They're based on commodity PC components and assembled in custom cases by various suppliers. You could buy the same computers if you wanted to.
Notice how all Netflix's computers are red? Netflix had their computers specially made to match their logo color.
From a software perspective, OCAs use the FreeBSD operating system and NGINX for the web server. Yes, every OCA has a web server. Video streams using NGINX. If none of these names make any sense, don't worry, I'm just including them for completeness.
The number of OCAs on a site depends on how reliable Netflix wants the site to be, the amount of Netflix traffic (bandwidth) that is delivered from that site, and the percentage of traffic a site allows to be streamed.
When you press play, you're watching video streaming from a specific OCA, like the one above, in a location near you.
For the best possible video viewing experience, what Netflix would really like to do is cache video in your house. But that's not practical yet. The next best thing is to put a mini-Netflix as close to your house as they can. How do they do that?
Where does Netflix put Open Connect Appliances (OCAs)?
Other video services, like YouTube and Amazon, deliver video on their own backbone network. These companies literally built their own global network for delivering video to users. That's very complicated and very expensive to do.
Netflix took a completely different approach to building its CDN.
Netflix doesn't operate its own network; it doesn't operate its own datacenters anymore either. Instead, internet service providers (ISPs) agree to put OCAs in their datacenters. OCAs are offered free to ISPs to embed in their networks. Netflix also puts OCAs in or close to internet exchange locations (IXPs).
Using this strategy Netflix doesn't need to operate its own datacenters, yet it gets all the benefits of being in a regular datacenter it's just someone else's datacenter. Genius!
Those last two paragraphs were pretty dense, so let's break it down.
Using ISPs to build a CDN.
An ISP is your internet provider. It's who you get your internet service from. It might be Verizon, Comcast, or thousands of other services.
The main point here is ISPs are located all around the world and they're close to customers. By placing OCAs in ISP datacenters, Netflix is also all over the world, and close to its customers.
Using IXPs to build a CDN.
An internet exchange location is a datacenter where ISPs and CDNs exchange internet traffic between their networks. It's just like going to a party to exchange Christmas presents with your friends. It's easier to exchange presents if everyone is in one place. It's easier to exchange network traffic if everyone is one place.
Each wire in the above picture connects one network to another network. That's how different networks exchange traffic with each other.
For Netflix, this is another win. IXPs are all over the world. So by putting their OCAs in IXPs, Netflix doesn't have to run its own datacenters.
Netflix has all this video sitting in S3. They have all these video serving computers spread throughout the world. There's just one thing missing: Video!
Netflix uses a process it calls proactive caching to efficiently copy video to OCAs.
A cache is a hiding place, especially one in the ground, for ammunition, food and treasures.
You know how squirrels bury nuts for the winter?
Each location they bury nuts is a cache. During the winter, any squirrel can find a nut cache and chow down.
Arctic explorers sent small teams ahead to cache food, fuel and other supplies along the route they were taking. The larger team following behind would stop at every cache location and resupply.
Both the squirrels and Arctic explorers were being proactive; they were doing something ahead of time to prepare for later.
Each OCA is a video cache of what you'll most likely want to watch.
Netflix caches video by predicting what you'll want to watch.
Everywhere in the world, Netflix, knows to a high degree of accuracy what its members like to watch and when they like to watch it. Remember how we said Netflix was a data-driven company?
Netflix uses its popularity data to predict which videos members probably will want to watch tomorrow in each location. Here, location means a cluster of OCAs housed within an ISP or IXP.
Netflix copies the predicted videos to one or more OCAs at each location. This is called prepositioning. Video is placed on OCAs before anyone even asks.
This gives great service to members. The video they want to watch is already close to them, ready and available for streaming.
Netflix operates what is called a tiered caching system.
The smaller OCAs we talked about earlier are placed in ISPs and IXPs. These are too small to contain the entire Netflix catalog of videos. Other locations have OCAs containing most of Netflix's video catalog. Still, other locations have big OCAs containing the entire Netflix catalog. These get their videos from S3.
Every night, each OCA wakes up and asks a service in AWS which videos it should have. The service in AWS sends the OCA a list of videos it's supposed to have based on the predictions we talked about earlier.
Each OCA is in charge of making sure it has all the videos on its list. If an OCA in the same location has one of the videos it's supposed to have, then it will copy the video from the local OCA. Otherwise, a nearby OCA with the video will be found and copied.
Since Netflix forecasts what will be popular tomorrow, there's always a one day lead time before a video is required to be on an OCA. This means videos can be copied during quiet, off-peak hours, substantially reducing bandwidth usage for ISPs.
There's never a cache miss in Open Connect. A cache miss would be asking for a specific video from an OCA and the OCA saying it doesn't have it. Cache misses happen all the time on other CDNs because you can't afford to copy content everywhere. Since Netflix knows all the videos it must cache, it knows exactly where each video is at all times. If a smaller OCA doesn't have a video, then one of the larger OCAs is always guaranteed to have it.
Why doesn't Netflix just copy all their video to every OCA in the world? Its video catalog is way too large to store everything at all locations. In 2013, the video catalog for Netflix was over 3 petabytes; I have no idea how large it is today, but I can only assume it's significantly larger.
That's why Netflix developed the method of choosing which videos to store on each OCA using data to predict what their members will want to watch.
Let's take an example. House of Cards is a very popular show. Which OCAs should it be copied to? Probably every location because members worldwide will want to watch House of Cards.
What if a video isn't as popular as House of Cards? Netflix decides which locations it should be copied to in order to best serve nearby member requests.
Within a location, a popular video like House of Cards is copied to many different OCAs. The more popular a video, the more servers it will be copied to. Why? If there was only one copy of a very popular video, streaming the video to members would overwhelm the server. As they say, many hands make light work.
A video isn't considered live when it's copied to just one OCA. Netflix wants to be able to play the same content at the same time everywhere in the world. Only when there are a sufficient number of OCAs with enough copies of the video to serve it appropriately, will the video be considered live and ready for members to watch.
Daredevil Season 2 in 2016, for example, was the first time Netflix released all episodes of a show, on all devices, in all countries, at the same time.
Hosting OCAs: What's in it for ISPs?
Why would an ISP agree to put an OCA cluster inside their network? At first blush, it seems too generous, but you'll be happy to know it's rooted firmly in self-interest.
To understand why, we'll need to talk about how networks work. I know throughout this book we've said cloud services are accessed over the internet. That's not the case for Netflix, at least when watching a video. When using a Netflix app, it talks to AWS over the internet.
The internet is an interconnect of networks. You have an ISP that provides internet service. I get my internet service from Comcast. What that means is my house connects to Comcast's network using a fiber optic cable. Comcast's network is their network; it's not the internet, the internet is something else.
Let's say I want to do a Google search, and I type a query into my browser and hit enter.
My request to Google first flows over Comcast's network. Google isn't on Comcast's network. At some point, my request has to go to Google's network. That's what the internet is for.
The internet connects Comcast's network to Google's network. There are these things called routing protocols that act like a traffic cop, directing where network traffic goes.
When my Google query is routed onto the internet it's not on Comcast's network anymore, and it's not on Google's network. It's on what's called the internet backbone.
The internet is woven together from many privately owned networks that choose to interoperate with each other. The IXPs we looked at earlier are one way networks connect with each other.
What Netflix has done with Open Connect is placed its OCA clusters inside the ISPs network. That means if I watch a Netflix video I'll be talking to an OCA in Comcast's network. All my video traffic is on Comcast's network; it never hits the internet.
The key to scaling video delivery is to be as close to users as possible. When you're doing that you're not using the internet backbone. Requests are being satisfied on a local part of the network.
Why is this a good thing? Recall that we said Netflix already consumes more than 37% of the internet traffic in the United States. If ISPs didn't cooperate, Netflix would use even more of the internet. The internet couldn't handle all the video traffic. ISPs would have to add a lot more network capacity, and that's expensive to build.
Right now, up to 100% of Netflix content is being served from within ISP networks. This reduces costs by relieving internet congestion for ISPs. At the same time, Netflix members experience a high-quality viewing experience. And network performance improves for everyone.
Earlier we discussed how Netflix increased the reliability of its system by running out of three different AWS regions. The architecture of Open Connect accomplished the same goal.
What may not be immediately obvious is that the OCAs are independent of each other. OCAs act as self-sufficient video-serving archipelagos. Members streaming from one OCA are not affected when other OCAs fail.
What happens when an OCA fails? The Netflix client you're using immediately switches to another OCA and resumes streaming.
What happens if too many people in one location use an OCA? The Netflix client will find a more lightly loaded OCA to use.
What happens if the network a member is using to stream video becomes overloaded? The same sort of thing. The Netflix client will find another OCA on a better performing network.
Open Connect is a very reliable and resilient system.
Netflix handles failures gracefully because it controls the client on every device running Netflix.
Netflix develops its Android and iOS apps themselves, so you might expect them to control those. But even on platforms like Smart TVs, where Netflix doesn't build the client, Netflix still has control because it controls the software development kit (SDK).
A SDK is a set of software development tools that allows the creation of applications. Every Netflix app makes requests to AWS and plays video using the SDK.
By controlling the SDK, Netflix can adapt consistently and transparently to slow networks, failed OCAs, and any other problems that might arise.
Netflix can be divided into three parts: the backend, the client, and the CDN.
All requests from Netflix clients are handled in AWS.
All video is streamed from a nearby Open Connect Appliance (OCA) in the Open Connect CDN.
Netflix operates out of three AWS regions and can usually handle a failure in any region without members even noticing.
New video content is transformed by Netflix into many different formats so the best format can be selected for viewing based on the device type, network quality, geographic location, and the member's subscription plan.
Every day, over Open Connect, Netflix distributes video throughout the world, based on what they predict members in each location will want to watch.
You select a video to watch using a client running on some device. The client sends a play request, indicating which video you want to play, to Netflix's Playback Apps service running in AWS.
We've not talked about this before, but a big part of what happens after you hit play has to do with licensing. Not every location in the world has a license to view every video. Netflix must determine if you have a valid license to view a particular video. We won't talk about how that works—it's really boring—but keep in mind it's always happening. One reason Netflix started developing its own content is to avoid licensing issues. Netflix wants to release a show to everyone in the world all at the same time. Creating its own content is the easiest way for Netflix to avoid worrying about licensing problems.
Taking into account all the relevant information, the Playback Apps service returns URLs for up to ten different OCA servers. These are the same sort of URLs you use all the time in your web browser. Netflix uses your IP address and information from ISPs to identify which OCA clusters are best for you to use.
The client intelligently selects which OCA to use. It does this by testing the quality of the network connection to each OCA. It will connect to the fastest, most reliable OCA first. The client keeps running these tests throughout the video streaming process.
The client probes to figure out the best way to receive content from the OCA.
The client connects to the OCA and starts streaming video to your device.
Have you noticed when watching a video the picture quality varies? Sometimes it will look pixelated, and after awhile the picture snaps back to HD quality? That's because the client is adapting to the quality of the network. If the network quality declines, the client lowers video quality to match. The client will switch to another OCA when the quality declines too much.
That's what happens when you press play on Netflix. Who would have ever thought so simple a thing as watching a video was so complex?
> Other video services, like YouTube and Amazon, deliver video on their own backbone network. These companies literally built their own global network for delivering video to users. That's very complicated and very expensive to do.
> Netflix took a completely different approach to building its CDN.
> Netflix doesn't operate its own network; it doesn't operate its own datacenters anymore either. Instead, internet service providers (ISPs) agree to put OCAs in their datacenters. OCAs are offered free to ISPs to embed in their networks. Netflix also puts OCAs in or close to internet exchange locations (IXPs).
This is false. Google has this too and it's called Google Edge Network. https://peering.google.com/ They have servers called Google Edge Nodes located in ISPs too.
I really liked this article, so fascinating. It was so well written, that I just had to buy the entire book.
That's interesting morsik, I haven't heard about Google doing the same thing. Do you have reference? I thought others might also embed in ISPs, but I didn't see anyone else publishing the details.
Extremely simple way to explain some complex setups. Netflix is innovative in all aspects.
I have not read anything that interesting in a long while. Thank you.
Very well written article.. Good to know such a vast architecture in play, behind the simple 'play' button..
Great article. Well thought through and detailed. Thanks for sharing this information.
Interesting read and really like your approach of explaining complex topic in simplicity. I was hooked till the end. Will look forward to reading more.
I'm so curious about AWS.
If netflix had a strong dependencies on AWS - which is part of Amazon - meanwhile amazon just tried to compete them with Amazon Prime, is it like a contrary when you stored your system/alghorithm right on your competitor?
Great Article. You are clearly explain network and video streaming.
1. Netflix plays more than 1 billion hours of video each week. As a comparison, YouTube streams 1 billion hours of video every day while Facebook streams 110 million hours of video every day.
From this, one can infer that YouTube has 7x the traffic of Netflix across the world. It does not say anything about the US numbers. Presumably YouTube has a much wider audience than Netflix, maybe 10x?
2. Netflix accounts for over 37% of peak internet traffic in the United States.
This only talks about peak traffic in the US. It does not directly correlate with the first statement.
Wonderful read and very informative.
Our consulting project has been telling this story to many of our clients for the past 18+ months. Especially the part about how everything is done in AWS ... right up until the user hits the "Play" button ... and then it's a mutually supportive hybrid cloud between the three key elements, 1-client, 2-AWS backoffice, & 3-CDN near the clients!
I love it. thank you for an amazing article.I just don't get the difference between AWS servers and OCAs. Why not just serve movies straight from AWS instead of caching them into OCAs?
1. Better performance. OCAs are in the same network as users so the video is closer to the user and doesn't have to transit the internet. This reduces latency, jitter, and increases bandwidth.
2. It also reduces traffic on the internet.
3. It gives them more control. The OCAs are finely tuned from a hardware and software perspective to deliver huge streams of video efficiently. They would not have that same control in AWS. For video I imagine this degree of control is a competitive advantage whereas for the backend stuff for the UI AWS is good enough.
4. And I imagine it's cheaper, but I don't know that for a fact or even if it's important to them.
This all IMHO of course.
"That's what happens when you press play on Netflix. Who would have ever thought so simple a thing as watching a video was so complex?"
A Systems or Software Engineer experienced in streaming Big Data applications, one would hope.
Very good article. Never knew so many things happen behind the scene when we click on "Watch" :). Thanks to Netflix for great service and thank you Todd.
Great article! One thing: can you not be more certain that Nefflix ALWAYS intended to be a "over the net" distribution company because "net" is in their name.....a conscious strategy dating all the way back since IPO and even since Venture startup? They only distributed DVDs to tide the time!


Mod the sims dark ember wood flooring. Wood floor adhesive premier building solutions. Woodflooringtrends current trends in the wood flooring. Cleaning engineered hardwood floors tips in easiest way. Engineered hardwood flooring specialty store in anaheim, ca. Hardwood floors russell hardwood floors. How to care for hardwood floorspeaches#039;n clean.
Wood look tile: 17 distressed, rustic, modern ideas. 30 great pictures and ideas basketweave bathroom floor tile. 12x12 vinyl tile tile design ideas. Laminate vinyl floor decor. Pallet flooring cheaper than wood diy 101 pallet ideas. Best wet mop for laminate floor. Nordic wood dark brown wall and floor tile floor tiles. Floor sweep shredder plant floor kernic systems. Decorative floor vases 28 images buy decorative floor. Ground floor cast and crew nice houzz. Lloyd mats ultimat plush carpet floor mats for sale.
Auto accessories headlight bulbs car gifts zone tech. Ideas: outstanding famous knotty pine paneling with. Heart pine floors 16 knotty pine flooring stained dark. Knotty pine stain colors knotty pine for flooring and. 1 x 6 knotty pine flooring floor matttroy. 25 best knotty pine walls ideas on pinterest pine walls. Orange plain patchwork lace long sleeve floor length dress.


Joined in November 2016. Erik holds a B.Sc. in Industrial Engineering and Management from the Royal Institute of Technology in Stockholm and has also studied at the National University of Singapore. From 2015-2016 he worked at Morgan Stanley as an Analyst within the EMEA Technology Group, based in London.


Set to take place at National Stadium, KL Sports City (formerly known as Bukit Jalil National Sports Complex), the Great Eastern Viper Arena Bkt Jalil will light up the night of 13th of July 2019 as thousands of fitness enthusiasts rise up to the Challenge in tackling its +-6km route with 12 awesome unique obstacles.
Holding true to Viper Challenge's motto of "Leave No One Behind", participants are encouraged to join with family and friends as a team, but individuals can still participate in the fun.
Registration opens on the 21st February 2019 at 12:01am and you can sign up as a team or individual and pick a start wave between 5:00pm -and later. Don't miss out on this amazing experience!
LIMITED TICKETS AVAILABLE so get in quick to avoid disappointment.
Great Eastern Viper Arena. You will see darkness in a whole new light.


text:
Severn Trent to hold convention to recruit plumbers and trainees for £78million pipe replacement scheme in Coventry
By On Jan 9, 2022
Plumbers are invited to a convention to apply for work on a £ 78 million pipe replacement project across Coventry.
Over the next three years, Severn Trent will replace 25,000 customer-owned lead supply pipes as part of its Green Recovery program.
The doors of Severn Trent's Academy in Finham, Coventry will open on Friday January 14th for plumbers and prospective plumbers to learn more about how to participate in the program.
The company can leave the qualification through the WatrerMark program to unskilled plumbers,
Laura Bingham of Severn Trent said, "We know how difficult times are for many due to the pandemic, so when we were creating this work program we knew we wanted to involve as many local plumbers and businesses as possible.
"The goal of this session is to work with as many local plumbers as possible who would like to be watermarked if they aren't already, and to understand how we can work together to replace utility lines from leading customers, which can be especially harmful to babies and children.
"This is a really exciting and ambitious project – so while we're doing the sharing with our own teams, we really want to give other installers and companies the opportunity to do this job where Severn Trent will be an essential part of the job means that the customer replaces his pipe and we support local companies. "
Severn Trent teams will be attending the academy and anyone wishing to attend can receive more information via email [email protected].
Severn Trent has started rolling out its trial in the Henley Green (CV2) neighborhood of the city, where residents initially expressed their interest.
The teams then conduct a tour of the property to arrange a replacement if the property meets the required criteria.
Severn Trent will seek to expand the area where the pipes will be replaced starting March next year and open the pilot to those with a Coventry zip code (CV1 through CV6).
The Coventry work is part of Severn Trent's broader $ 566 million Green Recovery program.
Individuals who want to learn more and see if their property is an option for replacement should visit here .
Like or follow our Facebook page – facebook.com/CovObserver for daily news updates straight to your news feed.
78millionconventionCoventryholdPipePlumbersrecruitreplacement
Kitchen Faucets Market Trend 2021 Forecast by Top Key Players
Malaysia Automotive Leaf Spring Assembly Market 2022 Analysis by Latest Trends, Future Growth And Keyplayers

redpajama_set_name: RedPajamaCommonCrawl


The physicians at Lowcountry Urology Clinics of Charleston SC have years of experience in the detection and treatment of conditions associated with enlarged prostate. This is also known as benign prostatic hyperplasia (sometimes as benign prostatic hypertrophy,) or BPH. Here are answers to a few commonly asked questions about enlarged prostate.
The prostate is a gland that surrounds your urethra, which is the tube that moves urine out of your body.
When the prostate gets bigger, it may partially squeeze or block the urethra. This may make it harder to urinate.
A normal part of getting older, BPH is caused by hormonal changes and cellular growth. You can't prevent it from developing.
Who typically gets an enlarged prostate?
Most men acquire BPH as they age. Around half of all men over 75 have an enlarged prostate.
You may have trouble "getting started," and find that the dribbling takes longer to stop when you are finished. You may often have the urge to urinate, which might wake you up at night. Your stream may seem weak. Or, when you are finished, your bladder may not empty completely.
Because it affects your ease of urination, an enlarged prostate can be very annoying. It is not a life-threatening condition, though.
BPH may be preventable or delayed with an appropriate lifestyle. It is treatable with several medications which improve the urine flow and decrease the troublesome symptoms. The doctor can also suggest changes in habits and lifestyle that can help relieve your inconvenience. If changes in habits and medications are not successful there are minimally invasive surgical procedures that can resolve the problem.
Treat the Symptoms of Enlarged Prostate at Lowcountry Urology Clinics Today!
Do you recognize the symptoms of BPH listed above? If so, realize that it's a normal part of aging, and that you can get some relief with help from your Urologist. So contact Lowcountry Urology Clinics of Charleston SC, and start feeling better!


Remember at the beginning of the week when you only had to keep track of who was being kicked off The Bachelor, and not a minute-by-minute play of who is currently our Prime Minister?
READ MORE: FOLLOW OUR LIVE BLOG.
And in this, the first of his weekly Friday installments of Joe-splaining, he's got some rather forceful advice for any MPs who are slowly murdering Australian democracy with their back-stabbing tendencies.


text:
Watch as the sled dogs race to the finish
Bachelor Butte Dog Derby
Wanoga Sno-Park, Cascade Lakes Hwy, Bend, OR 97703
The Bachelor Butte Dog Derby is a trophy dog sled race held on the first weekend of March at the Wanoga Sno-Park outside of Bend, OR. The event consists of 25 miles per day with seven – thirteen dogs. The event has something for everyone, from beginners to experienced long distance mushers. Sled dog classes include one and a half mile, five mile/five dog, nine mile/nine dog, and fifteen mile/seven dog races. Skijoring races will also be conducted on separate tracks, with one and a half mile, five mile, and nine mile classes. The Bachelor Butte Dog Derby is an opportunity to experience the splendor of the Cascades and contest one's dog team on world class terrain. The courses will be challenging and the classes offered somewhat nostalgic. Dog sled racing involves the timed competition of teams of sled dogs that pull a sled with the dog driver or musher standing on the runners. The team completing the marked course in the least time is judged the winner.
Upcoming Events Around Town
Author Event: What's Up with White Women by Ilsa Govan and Tilman Smith
Please join us for this virtual event. What's Up with White Women? is a guide for white women to strengthen their anti-racism practices. A powerful model using self-reflection and real-life stories of white women's experiences with sexism and white privilege gives them the tools to cultivate genuine partnerships with Black, Indigenous, and People of Color. As a white woman, ask yourself: are you upholding or fighting racism? What's Up with...
The Suttle Lodge & Boathouse
Suttle Lodge's Fireside Series
Join us for live music most Thursdays! Check out our happenings page for more information and to purchase tickets. All ages welcome. Tickets are FREE for guests staying overnight at the Lodge with RSVP. Please email info@thesuttlelodge.com to RSVP within 72 hours of show. Proof of vaccination and picture ID required at the door with no exceptions. Doors at 5:30. Show starts at 6pm.
Awareness Event: Just Say "Yes" to Life! Stories of Thriving After Stroke
In this one-of-a-kind collection of stories about surviving and thriving after stroke, people from all over the U.S. tell it like it is about the tragedies and triumphs they experienced after stroke. They speak with candor about their worst days, their grit and resilience, and the profound contentment and unexpected bursts of enthusiasm they feel for life today. Written to give solace to current stroke survivors, the 26 stories in...
Cascades Theatrical Company
January 20, 7:30 pm - 10:00 pm
Agatha Christie's Murder on the Orient Express
Ken Ludwig's clever adaptation of the Agatha Christie classic boasts all the glamour, intrigue and suspense of Dame Agatha's celebrated novel, with a healthy dose of humor to quicken the pace. Directed by Robert Flanagan Presented by arrangement with Concord Theatricals on behalf of Samuel French, Inc. www.concordtheatricals.com
Riverhouse Convention Center
Cascades Wedding Show
We're thrilled to announce the return of the Cascades Wedding Show! This one-day event gives Central Oregon brides and grooms the chance to connect with regional event professionals to help plan their big day. Join us on Saturday, January 22, 2022, from 10 am-3 pm at the Riverhouse on the Deschutes!
Midtown Ballroom
Midtown Events brings Big Head Todd and The Monsters to Midtown Ballroom Sunday, January 23 for an all ages show! Doors open at 6pm. Tickets on sale now at: https://www.eventbrite.com/e/big-head-todd-and-the-monsters-tickets-194156516367 Big Head Todd and The Monsters have quietly become an American institution following three and a half decades of writing, recording, and touring (totaling over 3,500 performances). After countless sold out shows in amphitheaters and on the high seas, beaming...
Volcanic Theatre Pub
Artikal Sound System Performing Live at Volcanic Theatre Pub
Based in South Florida, Artikal Sound System brings their R&B twist to the Cali reggae-rock genre on the newest album Welcome to Florida. The new album is due out February 1, 2022 on Controlled Substance Sound Labs. Welcome to Florida will be available everywhere you stream music. Pre-Save and Pre-Order here: https://cssl.fanlink.to/welcome-to-florida
Author Event: These Bones by Kayla Chenault & The Salt Fields by Stacy D. Flood
Please join us for this in-conversation virtual event. These Bones by Kayla Chenault Chenault's mesmerizing debut follows the Lyons family starting in 1909 through a series of vignettes that create a mosaic portrait of the Bramble Patch, the Black section of fictional Midwestern city Napoleonville, which was "built on the bones of its dead." The tale is framed by a request for information from the Napoleonville Historical society to Wanhope...
Deschutes Land Trust
Mountain snowpacks have historically acted as large, natural reservoirs of water, as well as providing awesome recreational opportunities. In recent decades, however, snowpack has declined—another sign of a changing climate. If our climate continues to warm, snow loss will be exacerbated across the Western US, termed a "low-to-no snow future." Join the Deschutes Land Trust, Dr. Alan Rhoades, and Dr. Erica Siirila-Woodburn to learn about the possibility of a low-to-no...


We demonstrate that second harmonic generation is a sensitive probe of thin-film deposition conditions.
Hill, Kenneth D.. , Woods, Donald J..
Zhang, Jue. , Sheldon, P.. , Ahrenkiel, R. K..
Collins, R. W.. , Cavese, J. M..
VanZandt, T.. , Browning, R.. , Lee, S. Y.. , Khan, M. R.. , Pressesky, J. L.. , Duan, S. L.. , Heiman, N..
Johnson, K. E. Thin-film recording media: Challenges for physics and magnetism in the 1990s (invited) : 35th annual conference on magnetism and magnetic materials.


text:
ACCEPTED

#### According to
The Catalogue of Life, 3rd January 2011

#### Published in
Gen. Amer. bor. 1:76. 1848

#### Original name
null

### Remarks
null

redpajama_set_name: RedPajamaGithub


Check out Jumbo Quinn's website for great pics of the Pirate Band at this years football games.
KEOM Radio did a "Spot On" segment featuring the Pirate Band! Check it out by clicking here.
Check out these fantastic preofessional photos from the sideline in San Antonio! Click here to visit the Jolesch website. Click the pull down menu under "My Category is...." and choose Poteet HS. Enjoy!
Mr. Video has all the prefoessional quality video from State Marching Contest you would ever like to see. See the Pirate Band in action from the hi cam or multi-view, blu-ray, etc. Click here for more details.


ProactivePack is a BMC MarketZone and Technology Alliance partner company established in France.
ProactivePack was founded by two IT event management experts, totalizing more than 30 years experience in the area of IT monitoring, in particular around BMC solutions. Both founders work daily, in Europe and beyond, for large corporations with the highest demands in terms of monitoring and industrialization.
Pragmatic and customer focused, they consolidated their experiences and numerous customer feedback to build a companion product for BMC TrueSight,BPPM and BEM solutions.


Our naturopath gave me this recipe during our appointment for Chloé the other day. It was almost worth the price of the consultation just to get this recipe ;) These granola bars are one of the tastiest snacks I have ever eaten. This also happens to be an allergy/intolerance-friendly recipe that is packed to the brim with all things healthy.
I don't really consider myself to be a crunchy mama. But I came across this video on youtube earlier this week (as well as the follow-up video) and laughed as I found myself identifying with more than a few things. Then I took a peek on-line at EcoParent, which I didn't know about at all, and found myself wanting to get a subscription.
Hmm.... maybe I'm a little crunchier than I thought???
After all, I buy into alternative therapies, such as osteopathy and naturopathy. I breastfed both kids, wore them in slings, co-slept and used cloth diapers until their sensory issues became evident. And though I do find myself on occasion using processed foods for convenience when life is just too busy with the kids, I am definitely all for eating as little pre-packaged and processed foods as possible. And I get super excited over recipes such as these with all sorts of random superfoods.
Yup. Probably a little bit crunchy... lol.
After getting this recipe from our naturopath, I walked into our house, went straight to the kitchen and immediately got to work on these. That's just how yummy I thought they were going to be.
These are a little more time consuming to make than other granola bar recipes, such as my no bake hemp seed granola balls. You've got to roast a few of the ingredients and later, melt a few things together. It's not terribly tough, but just a few more steps than many other traditional granola bar recipes.
But let me tell you... these are soooooooooooooooooo worth the effort. I think these must be the best gosh-darned granola bars I have ever tried in my life.
Now, I'm usually pretty able to make delicious treats, have one, and then leave them in the kitchen without further obsessive temptation. For instance, I can control myself without effort around peanut butter brownies, even though I LOVE peanut butter. Same with some of my favourite sinful muffins, such as muffins that taste like donuts and maple syrup muffins.
I think I had two of these bars in the afternoon before picking the kids up from daycare. And then another upon my return. And I then had to use every ounce of willpower in me to prevent myself from having a fourth before going to bed.
Even Logan is in LOVE with these. Yesterday, I packed two HUGE pieces for our visit to the hospital for his eye blinking. I figured that that would be plenty of food for both him and myself, even if we got stuck there a long time.
He had polished off BOTH of the huge squares by 10:30 a.m., barely leaving me a bite!!! And each was a big portion even for me!
Ah well... This'll be a great way to stuff him full of healthy things.
Because not only are these delicious. But they couldn't be healthier if you tried. Flax, chia, hemp... what else could you ask for??
Now, I'm not sure what exactly makes these so darned delicious. So I'll tell you how I made mine so that you can replicate. Or choose to do differently. Whatever suits your fancy.
As an aside, this is the first time that I've ever used coconut oil (which I had coincidentally picked up at the health food store just a few days earlier). So I'm not sure if that's part of why they were so good too.
dried cherries because I didn't have goji berries or dried cranberries. Mmm... I think that the mix of slightly salty with the sweetness of the dried cherries is part of what I loved so much!
chocolate chips, because I didn't even know what cocoa nibs were until googling them!
I omitted the lemon zest, only because Jérôme doesn't love lemon and I thought it kinda sounded weird in a granola bar anyway.
Just a quick note also, that I get my hemp, chia and ground flax at Costco. Which is also where I had gotten my dried cherries.
1. Preheat oven to 350 degrees Farenheit. Line an 8 x 12 inch baking dish with parchment paper.
2. Toss the oats, sunflower seeds, and coconut in a bowl and spread out on a dry sheet pan and toast for 10 minutes, stirring occasionally, until lightly browned. (Don't skip this step - they're not nearly as tasty if you don't toast everything).
3. In a small pot, melt the coconut oil, maple syrup, coconut sugar, and salt on medium heat until it boils. Stir for one minute and take off heat.
4. Add vanilla and toss over oat mixture, adding all the rest of the ingredients, and combine well. (Note: the chocolate chips melt into the mixture as you stir. If you prefer they be whole, wait for the mixture to cool before adding chocolate chips. Though I liked it this way, as it gave the entire bar a bit of a chocolatey flavour).
5. Pour mixture into parchment lined 8 x 12 inch baking pan.
6. Bake at 300 degrees Farenheit for about 20 minutes, until golden. (Note: I cannot read directions and only noticed the temperature change while typing. I baked mine at 350 degrees and love how they came out a bit crunchy. But if you prefer less crunchy bars, you'd be better to follow the original directions).
7. Let cool in pan before cutting in squares. (Or just scoop it all directly into your mouth!).
Tip: If using the goji berries, try eliminating the lemon zest for a slightly more conventional granola bar flavour.
Sweet i knew something about food that yiu didn't, haha. I'm a bit of a cocoa nib nut... Eat em wuth dried cherries and sunflower seeds in trail mix.
Totally trying these... Thanks for sharing!!
I haven't, but I'm almost positive they'd be fine. I've frozen lots of other types of homemade granola bars.
P.S. You can always check my tags at the bottom of posts, because I tag posts as freezer-friendly.
I promise you... these will NOT disappoint!


Campus Book Rentals ....Save on Text books.
Going back to school costs enough, why not save where you can?
Another great idea is if there is a supplementary book that you are interested in but not sure you want to put out that kind of money for it rent it first!!


text:
Who has the REAL Religious Freedom argument?
Posted by Aclu Texas 20sc on February 16, 2016
Religious freedom does not give any of us the right to discriminate against or harm others. Learn the difference between real threats to religious liberties and using religion as an excuse to attack others.
Correct: A. As more people gain greater equality under the law, we are seeing a troubling push to allow anyone, including business owners and public officials, to use their religious beliefs as an excuse to discriminate and impose their religious beliefs on others. These efforts undermine our core American values of fairness and equality—and they are harmful, wrong and unconstitutional. Everyone, including our LGBT brothers and sisters, should be treated fairly and equally under the law.
Incorrect: B. Through comments and official correspondence, Texas Gov. Abbott, Lt. Gov., and AG Paxton have suggested that it's ok for county clerks avoid their duty and refuse to issue licenses to same-sex couples if they held religious objections. They're wrong. The first duty of public officials is to uphold the law, even if doing so conflicts with their personal religious convictions. And in the wake of the Supreme Court's ruling in Obergefell v. Hodges, marriage equality is now the law of the land.
Our personal religious beliefs don't give us a special exemption from following laws meant to protect everyone. Imagine what would happen if each of us could choose, according to our individual beliefs, which laws to follow and which to ignore?
Correct: A. For many Sikhs, the turban is sacred and never removed in public. You have the right to wear a religious head covering during a security check. Security might insist on patting it down, but you still have the right to keep it on, unless security thinks there is additional cause for concern. See: https://www.aclu.org/know-your-rights/when-faced-anti-muslim-discrimination.
Incorrect: B. Under the First and Fourteenth Amendments to the U.S. Constitution and Texas' Religious Freedom Restoration Act, students' right to wear articles of faith in school is indisputably protected. Public school officials can't prohibit students from wearing rosaries or crosses visibly at school.
Correct: A. As more people gain greater equality under the law, we are seeing a troubling push to allow anyone, including business owners and public officials, to use their religious beliefs as an excuse to discriminate and impose their religious beliefs on others. These efforts undermine our core American values of fairness and equality—and they are harmful, wrong and unconstitutional. Everyone is entitled to his or her own religious beliefs, but that does not mean, and has never meant, that businesses—including pharmacists—can use religion as an excuse to deny a woman birth control or other reproductive health care needs.
Correct: A. Students in school have a right to pray or share messages expressing their religious beliefs alone or with friends at any time that does not disrupt classes—even when that speech is by students as young as elementary schoolchildren. However, schools cannot require students to participate in prayer or organize prayer activities, and all prayer in school must be student-initiated. Students have the right to pray in school. Schools and school officials do not have the right to coerce students into praying in school.
Correct. A. Under the First and Fourteenth Amendments to the U.S. Constitution and Texas' Religious Freedom Restoration Act, students' right to wear articles of faith in school is indisputably protected. That includes the right of a Native American elementary school student to wear his hair in the manner consistent with his religious beliefs.
Incorrect. B. No, Texas State Board of Education, this is just wrong-headed and stupid. See http://tfn.org/whats-in-the-proposed-new-texas-textbooks-we-got-democracy-from-moses/ and http://tfn.org/no-the-constitution-isnt-founded-on-the-ten-commandments/
Correct: A. Equality and religious freedom are our bedrock values. They are enshrined in our Constitution and are not mere formalities to be discarded in difficult times. Yet recently, we have seen a politics of fear used to justify discrimination against Muslims. This has resulted in unwarranted surveillance, unlawful profiling, and exclusionary immigration policies targeting people based on their faith, nationality, or national origin. Targeting mosques and the free expression of religion is just plain wrong and unconstitutional. In the U.S., you have the right to practice your faith freely, and you are protected against government intrusion into matters of faith.
https://www.aclu.org/feature/anti-muslim-discrimination
Take Action: Tell Texas leaders religion should never be used to discriminate against or harm others.


text:
Arachnanthus  è un genere di antozoi della famiglia Arachnactidae.

Tassonomia
Comprende le seguenti specie:
Arachnanthus australiae
Arachnanthus bocki
Arachnanthus lilith
Arachnanthus oligopodus ()
Arachnanthus sarsi

Note

Altri progetti

Collegamenti esterni

Ceriantharia

redpajama_set_name: RedPajamaWikipedia


Finding the USB-C AV Multiport adapter in stores yet?
Discussion in 'MacBook' started by negatv1, Apr 16, 2015.
Is anyone finding the USB-C AV Multiport adapter in stores yet? Apple/Best Buy or otherwise. Showing up only as ship to store/4-6weeks at stores near me.
Same here I visited two Apple stores and 2 independent stores, no stock, ETA. I did pickup the USB-C to full size USB though. All stores had those, and charging cables.
no multiport adapter, but they do have plenty of stock on the rest, charger, charging cable and usb adapter.
Already have the Macbook, but foolishly only ordered the USB-C to A adapter at the time of ordering. Eager to spend that $80 I suppose.
they do have plenty of stock on ... usb adapter.
I can't find any USB-C adapters on BB's website. Can you share the direct link? Thanks!
i was talking about the apple store, physical apple store. sorry for the confusion.
I would skip the adapter and get the kickstarter hydadock. Seems like a better value for the money.
Quite a difference in price between the black and white ones.
I agree the Hydradock looks nice, but it doesn't really exist as of yet. In theory the Apple adapters exist... somewhere.
I think I got lucky. I received my adapter on April 13 but had to wait until the 16th for my Macbook. Too bad my Macbook was DOA. Now I have to wait weeks to get another one. Adapter but no macbook.... just great..
I'm almost certain that Hydradock is vaporware. Check out the other thread about it.
I'm feeling really lucky that I got my USB-C to USB adapter and my AV Multiport adapter shipped!
The thing is, if you call the store, it seems you get some operator that isn't actually IN the store. Maybe I'm wrong, but every day this week that I called, they said that it needs to be bought online and they didn't have any in stock in all of San Diego. I called this morning, the same story... I stop by after work, they had 2 in stock. I bought both of them and cancelled my order online which was still pending to be delivered "April 29th - some day in May".
The operators are not in the store. I asked. However, they were willing to call the actual store for me when I asked nicely. When I went to the store to inquire, they just load up the website and use that as their stock check. When I mentioned that I bought one earlier in the week when it indicated not in stock through the website, the rep was willing to look in the back.
Sounds like the trick is to kindly get the operator to put you through to the actual store and have someone there physically check inventory, since the online store gives the same 'inventory' results that we see online.
If you dial the store and when it asks how it can help, say "manager" it connects you to the actual store.
I ordered an adapter at 2am on launch day, and still didn't have a CC charge yet. I found a MacBook at BB, but really needed the adapter. I checked online and called, and was told no stock.
Following the advice in MacRumors, I went to my nearest Apple store (Plano, TX, Willow Bend store) and they had two left. The guy said they got in about 20 and they were going fast. He asked if I wanted both, and I thought about it a minute or so, but that would be too expensive. He said once they are gone he doesn't expect to get any more for a long time. Not sure why, though. So, I then cancelled my Apple store order.
I'm now able to run my desktop 1920x1200 monitor and external HD while charging. I'm using a Bluetooth keyboard and mouse.
how does the macbook run an external monitor?
Performance is very good, I don't sense any slowdown. I think most of the performance problems people have been reporting with the rMB are caused by retina scaling and HiDPI mode, which are not enabled for external monitors.