Learn how to manage costs in Firebase with both clientside and serverside optimizations. https://angularfirebase.com/lessons/managing-firebase-costs/

Source Article https://hackernoon.com/how-we-spent-30k-usd-in-firebase-in-less-than-72-hours-307490bd24d
Firebase Pricing https://firebase.google.com/pricing/
2024-02-04T07:23:50.902Z
nova-2
2-general-nova
2024-01-09.29447
30089e05-99d1-4376-b32e-c263170674af
ccc646b3-2733-4355-a5c3-af2171fb8609
00eb1ac6195153fec359c951d067e1ddf0dae667eca8924ad9bcaac6e77549da
67875a7f-c9c4-48a0-aa55-5bdb8a91c34a
deprecated
Imagine if you woke up one day and your Firebase bill went from 0 to 30,000 US dollars.
Well, that's exactly what happened to a startup in Colombia that scaled up to about 2,000,000 daily active users.
They really just made one seemingly minor mistake.
And with this video, I wanna show you how we can take their $30,000 bill and reduce it down to about $25.
If you're new to the channel, make sure to like and subscribe, and you can grab the source code for this project at angularfirebase.com.
So you may have already seen this article because it kind of went viral last week on Hacker Noon.
But the source of this high bill was because they managed to make over 40,000,000,000 requests to Firestore in less than went 48 hours.
So how is that even possible when you only have 2,000,000 users?
That means every user would have to make about 20 was in Document Reads.
The mistake they made is that they needed to calculate the total for every crowdfunding campaign, but they were doing this buy reading every single document in a collection for every user on the site.
So as they got more users and more payments, the problem just compounded to result in a load of document reads.
Not only is that super expensive, but it also results in a lot of latency on the client and that's actually how they discovered the problem in the first and I should also point out that this type of mistake is very common and you really won't notice it until you start scaling up end users and you run into these problems with billing and performance.
Minutes.
I'm gonna show you how to avoid this with strategies both client side and server side.
But the number one thing you should do right off the bat is set up a billing alert on Google Cloud Platform.
Every Firebase perform.
Every Firebase project is tied to a GCP project.
So you just have to go into the billing panel and then set up the billing thresholds that you wanna be alerted on.
So that way you can find out early if you have some kind of issue that's resulting in billing spikes.
But that doesn't really solve the problem at hand because the problem is in the code.
So I set up a demo here that does something similar.
Basically, it calculates the total amount of donations, the average amount of each donation, and this is all based on a Firestore collection.
Should.
But I have this app super poorly optimized where I'm reading every document in the collection 3 separate times, one for the total, one for the average, and then one to show the list of donations.
So if this collection has 100 documents in it, it will take 300 document reads to present the UI that you see on the left.
Believe it or not, this is pretty easy to do and it still performs great at this level so you really wouldn't notice an issue at this point.
The startup in the article was using Angular so we'll be doing the same here.
Will be using Firestore with AngularFire 2 and then I'm importing Faker so I can generate some fake data for the UI.
Now the first thing I want to show you is how we can optimize things client side and I think that's really where you should focus first.
So we have 3 different observables hear 1 for the donations, 1 for the total amount, and 1 for the average amount.
So the mistake that I'm making here first is that I'm creating a separate observable for the same collection went to show all 3 of these different amounts.
That means that we're going to subscribe to 3 different observables, fire off 3 separate read operations to Firestore even though they all depend on the same source data.
I'm not really going to explain this code in detail, but basically what we're doing is taking the Firestore collection, which is an array, and then we reduce it down to the total amount of all of the donations in that collection.
And then we do the same thing for the average with just a slightly different calculation.
And then we subscribe to all 3 of these observables in the HTML using the async pipe.
So the main problem here is that we have 3 different data sources that are being created even though we could really get this all done with a single data source.
So let's take a look at how we might use use JS to reduce our document reads down from about 300 to 100.
What we wanna do is have one observable that can be shared with the other observables that make the calculation for the total and the average.
So what I'm going to do first is pipe in the tap operator so we can see when a subscription is created to this observable.
Then we'll also console log the number of documents to get an approximate amount of the reads that were executed.
Did.
So now instead of having the total and the average create their own observables, we'll just go ahead and use the donations observable that we created above.
So let's take a look and see if that actually solved the problem in the UI.
We should only get one console log.
But if we look closely here, we're still getting 3 side effects which should mean that we're sending off 3 sets of reads to Firestore.
Luckily, there's an easy way to fix this with RxJS and that's just by piping in the share get a replay operator which will make this a hot observable and allow any other subscribers to always get the last cached value.
So now we can be a 100% confident that our code is only sending off one document read to the back end.
So that can be a super useful technique, and you can also use an Angular service to inject the same data source into multiple components.
But the actual problem that was faced by the start is that we need all of the information from every document in that collection.
And to do that client side means that we would have to read every single document every time which is just not feasible.
Want, but we can implement a server side solution to make this way more efficient and we can do it with Firebase Cloud Functions.
An app like this is very heavy on document reads but were very light on document rights.
For example, we might have a 100 people visit the page, but only one of those people actually make a donation and write something to the database.
One of the main reasons that NoSQL databases like Firestore are so popular is that we have the flexibility to optimize for read performance.
What we're going to do next is set up a Cloud Function that listens to the donation collection.
And every time a new document is written there, it will update an aggregated document that contains the total count, the total amount, and also the last 5 donations that were left.
So this means we could have a 1,000,000 documents in our donations use collection and have all the information needed for the front end UI with a single document read.
The actual data payload will now be much smaller so our front end UI is going to be way more performant for 99% of users.
And basically, what we're doing is trade in 20,000 client side document reads for an extra write and an extra Cloud Function invocation.
In this case, that's an excellent trade off and it'll make the pricing go from 30 k down to pretty close to the seen go from 30 k down to pretty close to the free tier.
One of the drawbacks is that it makes writes to the UI a little bit slower because we have to wait for that Cloud Function to finish before the document is updated.
But in this case, it makes a ton of sense because the vast majority of users are reading data and not changing it.
I have initialized Firebase Cloud Functions in this project, and I'm also using the new Node 8 runtime which allows me to use async await in just a plain JavaScript function.
Will bring in the admin database and then will want to listen to every new donation document that is created in that Firestore collection.
Want a new document is created, we'll just want to update the aggregated data for that collection.
So we're no longer displaying the actual donation documents, but instead were displaying the aggregated document based on that collection.
There is no best practice for this.
But, generally, what I do is I save a document in a collection called aggregation and the ID for that document matches the collection that we're making calculations on.
And you really just wanna save the bare minimum amount information you need to make calculations in the UI.
In our case, we just need to show the average and the total.
So we can calculate these by taking the current aggregated total and then add it to the amount of the new donation, and then we can take the total count and just add 1 to it.
And then we'll also save the last 5 donations so we can show those in the view as well.
By saving the last 5 donations here, we can show those in the view, and will only have to execute 1 document read instead of 5, which can make a pretty big difference if you have 2,000,000 users.
And then we'll complete our function by was returning a set operation from it, and we're good to go.
Our front end UI looks exactly like it did before, but instead, we're just doing a single document read instead of, say, 20,000.
The nice thing about Firestore is that it's easy to anticipate your pricing.
Let's imagine that every user reads about 10 documents per visit.
That would be about 20,000,000 document reads per 2,000,000 users or about $25 per 2,000,000 merchants or savings of about 99.9% from the startup's initial query setup.
I'm gonna go ahead and wrap things up there.
If if this video helped you, please like and subscribe.
And if you want to learn how to get a quality app shipped to production, consider becoming a pro member at angularfirebase.com.
You'll get a free copy of my book, access to my courses, and a whole bunch of other free stuff.
Thanks for watching and I'll talk to you soon.

Speaker 0: Imagine if you woke up one day and your Firebase bill went from 0 to 30,000 US dollars. Well, that's exactly what happened to a startup in Colombia that scaled up to about 2,000,000 daily active users. They really just made one seemingly minor mistake. And with this video, I wanna show you how we can take their $30,000 bill and reduce it down to about $25. If you're new to the channel, make sure to like and subscribe, and you can grab the source code for this project at angularfirebase.com.

So you may have already seen this article because it kind of went viral last week on Hacker Noon. But the source of this high bill was because they managed to make over 40,000,000,000 requests to Firestore in less than went 48 hours. So how is that even possible when you only have 2,000,000 users? That means every user would have to make about 20 was in Document Reads. The mistake they made is that they needed to calculate the total for every crowdfunding campaign, but they were doing this buy reading every single document in a collection for every user on the site.

So as they got more users and more payments, the problem just compounded to result in a load of document reads. Not only is that super expensive, but it also results in a lot of latency on the client and that's actually how they discovered the problem in the first and I should also point out that this type of mistake is very common and you really won't notice it until you start scaling up end users and you run into these problems with billing and performance. Minutes. I'm gonna show you how to avoid this with strategies both client side and server side. But the number one thing you should do right off the bat is set up a billing alert on Google Cloud Platform.

Every Firebase perform. Every Firebase project is tied to a GCP project. So you just have to go into the billing panel and then set up the billing thresholds that you wanna be alerted on. So that way you can find out early if you have some kind of issue that's resulting in billing spikes. But that doesn't really solve the problem at hand because the problem is in the code.

So I set up a demo here that does something similar. Basically, it calculates the total amount of donations, the average amount of each donation, and this is all based on a Firestore collection. Should. But I have this app super poorly optimized where I'm reading every document in the collection 3 separate times, one for the total, one for the average, and then one to show the list of donations. So if this collection has 100 documents in it, it will take 300 document reads to present the UI that you see on the left.

Believe it or not, this is pretty easy to do and it still performs great at this level so you really wouldn't notice an issue at this point. The startup in the article was using Angular so we'll be doing the same here. Will be using Firestore with AngularFire 2 and then I'm importing Faker so I can generate some fake data for the UI. Now the first thing I want to show you is how we can optimize things client side and I think that's really where you should focus first. So we have 3 different observables hear 1 for the donations, 1 for the total amount, and 1 for the average amount.

So the mistake that I'm making here first is that I'm creating a separate observable for the same collection went to show all 3 of these different amounts. That means that we're going to subscribe to 3 different observables, fire off 3 separate read operations to Firestore even though they all depend on the same source data. I'm not really going to explain this code in detail, but basically what we're doing is taking the Firestore collection, which is an array, and then we reduce it down to the total amount of all of the donations in that collection. And then we do the same thing for the average with just a slightly different calculation. And then we subscribe to all 3 of these observables in the HTML using the async pipe.

So the main problem here is that we have 3 different data sources that are being created even though we could really get this all done with a single data source. So let's take a look at how we might use use JS to reduce our document reads down from about 300 to 100. What we wanna do is have one observable that can be shared with the other observables that make the calculation for the total and the average. So what I'm going to do first is pipe in the tap operator so we can see when a subscription is created to this observable. Then we'll also console log the number of documents to get an approximate amount of the reads that were executed.

Did. So now instead of having the total and the average create their own observables, we'll just go ahead and use the donations observable that we created above. So let's take a look and see if that actually solved the problem in the UI. We should only get one console log. But if we look closely here, we're still getting 3 side effects which should mean that we're sending off 3 sets of reads to Firestore.

Luckily, there's an easy way to fix this with RxJS and that's just by piping in the share get a replay operator which will make this a hot observable and allow any other subscribers to always get the last cached value. So now we can be a 100% confident that our code is only sending off one document read to the back end. So that can be a super useful technique, and you can also use an Angular service to inject the same data source into multiple components. But the actual problem that was faced by the start is that we need all of the information from every document in that collection. And to do that client side means that we would have to read every single document every time which is just not feasible.

Want, but we can implement a server side solution to make this way more efficient and we can do it with Firebase Cloud Functions. An app like this is very heavy on document reads but were very light on document rights. For example, we might have a 100 people visit the page, but only one of those people actually make a donation and write something to the database. One of the main reasons that NoSQL databases like Firestore are so popular is that we have the flexibility to optimize for read performance. What we're going to do next is set up a Cloud Function that listens to the donation collection.

And every time a new document is written there, it will update an aggregated document that contains the total count, the total amount, and also the last 5 donations that were left. So this means we could have a 1,000,000 documents in our donations use collection and have all the information needed for the front end UI with a single document read. The actual data payload will now be much smaller so our front end UI is going to be way more performant for 99% of users. And basically, what we're doing is trade in 20,000 client side document reads for an extra write and an extra Cloud Function invocation. In this case, that's an excellent trade off and it'll make the pricing go from 30 k down to pretty close to the seen go from 30 k down to pretty close to the free tier.

One of the drawbacks is that it makes writes to the UI a little bit slower because we have to wait for that Cloud Function to finish before the document is updated. But in this case, it makes a ton of sense because the vast majority of users are reading data and not changing it. I have initialized Firebase Cloud Functions in this project, and I'm also using the new Node 8 runtime which allows me to use async await in just a plain JavaScript function. Will bring in the admin database and then will want to listen to every new donation document that is created in that Firestore collection. Want a new document is created, we'll just want to update the aggregated data for that collection.

So we're no longer displaying the actual donation documents, but instead were displaying the aggregated document based on that collection. There is no best practice for this. But, generally, what I do is I save a document in a collection called aggregation and the ID for that document matches the collection that we're making calculations on. And you really just wanna save the bare minimum amount information you need to make calculations in the UI. In our case, we just need to show the average and the total.

So we can calculate these by taking the current aggregated total and then add it to the amount of the new donation, and then we can take the total count and just add 1 to it. And then we'll also save the last 5 donations so we can show those in the view as well. By saving the last 5 donations here, we can show those in the view, and will only have to execute 1 document read instead of 5, which can make a pretty big difference if you have 2,000,000 users. And then we'll complete our function by was returning a set operation from it, and we're good to go. Our front end UI looks exactly like it did before, but instead, we're just doing a single document read instead of, say, 20,000.

The nice thing about Firestore is that it's easy to anticipate your pricing. Let's imagine that every user reads about 10 documents per visit. That would be about 20,000,000 document reads per 2,000,000 users or about $25 per 2,000,000 merchants or savings of about 99.9% from the startup's initial query setup. I'm gonna go ahead and wrap things up there. If if this video helped you, please like and subscribe.

And if you want to learn how to get a quality app shipped to production, consider becoming a pro member at angularfirebase.com. You'll get a free copy of my book, access to my courses, and a whole bunch of other free stuff. Thanks for watching and I'll talk to you soon.
Imagine if you woke up one day and your Firebase bill went from 0 to 30,000 US dollars. Well, that's exactly what happened to a startup in Colombia that scaled up to about 2,000,000 daily active users.
They really just made one seemingly minor mistake. And with this video, I wanna show you how we can take their $30,000 bill and reduce it down to about $25. If you're new to the channel, make sure to like and subscribe, and you can grab the source code for this project at angularfirebase.com.
youtube
So you may have already seen this article because it kind of went viral last week on Hacker Noon. But the source of this high bill was because they managed to make over 40,000,000,000 requests to Firestore in less than went 48 hours. So how is that even possible when you only have 2,000,000 users? That means every user would have to make about 20 was in Document Reads. The mistake they made is that they needed to calculate the total for every crowdfunding campaign, but they were doing this buy reading every single document in a collection for every user on the site.
So as they got more users and more payments, the problem just compounded to result in a load of document reads. Not only is that super expensive, but it also results in a lot of latency on the client and that's actually how they discovered the problem in the first and I should also point out that this type of mistake is very common and you really won't notice it until you start scaling up end users and you run into these problems with billing and performance.
billing
Minutes. I'm gonna show you how to avoid this with strategies both client side and server side. But the number one thing you should do right off the bat is set up a billing alert on Google Cloud Platform.
Every Firebase perform. Every Firebase project is tied to a GCP project. So you just have to go into the billing panel and then set up the billing thresholds that you wanna be alerted on. So that way you can find out early if you have some kind of issue that's resulting in billing spikes. But that doesn't really solve the problem at hand because the problem is in the code. So I set up a demo here that does something similar. Basically, it calculates the total amount of donations, the average amount of each donation, and this is all based on a Firestore collection.
billing
revenue
Should. But I have this app super poorly optimized where I'm reading every document in the collection 3 separate times, one for the total, one for the average, and then one to show the list of donations. So if this collection has 100 documents in it, it will take 300 document reads to present the UI that you see on the left. Believe it or not, this is pretty easy to do and it still performs great at this level so you really wouldn't notice an issue at this point. The startup in the article was using Angular so we'll be doing the same here. Will be using Firestore with AngularFire 2 and then I'm importing Faker so I can generate some fake data for the UI. Now the first thing I want to show you is how we can optimize things client side and I think that's really where you should focus first. So we have 3 different observables hear 1 for the donations, 1 for the total amount, and 1 for the average amount. So the mistake that I'm making here first is that I'm creating a separate observable for the same collection went to show all 3 of these different amounts. That means that we're going to subscribe to 3 different observables, fire off 3 separate read operations to Firestore even though they all depend on the same source data.
I'm not really going to explain this code in detail, but basically what we're doing is taking the Firestore collection, which is an array, and then we reduce it down to the total amount of all of the donations in that collection. And then we do the same thing for the average with just a slightly different calculation.
billing
revenue
sales
And then we subscribe to all 3 of these observables in the HTML using the async pipe. So the main problem here is that we have 3 different data sources that are being created even though we could really get this all done with a single data source. So let's take a look at how we might use use JS to reduce our document reads down from about 300 to 100. What we wanna do is have one observable that can be shared with the other observables that make the calculation for the total and the average.
So what I'm going to do first is pipe in the tap operator so we can see when a subscription is created to this observable. Then we'll also console log the number of documents to get an approximate amount of the reads that were executed. Did. So now instead of having the total and the average create their own observables, we'll just go ahead and use the donations observable that we created above. So let's take a look and see if that actually solved the problem in the UI.
programming
documentation
We should only get one console log. But if we look closely here, we're still getting 3 side effects which should mean that we're sending off 3 sets of reads to Firestore. Luckily, there's an easy way to fix this with RxJS and that's just by piping in the share get a replay operator which will make this a hot observable and allow any other subscribers to always get the last cached value.
So now we can be a 100% confident that our code is only sending off one document read to the back end. So that can be a super useful technique, and you can also use an Angular service to inject the same data source into multiple components. But the actual problem that was faced by the start is that we need all of the information from every document in that collection.
documentation
And to do that client side means that we would have to read every single document every time which is just not feasible.
Want, but we can implement a server side solution to make this way more efficient and we can do it with Firebase Cloud Functions. An app like this is very heavy on document reads but were very light on document rights. For example, we might have a 100 people visit the page, but only one of those people actually make a donation and write something to the database. One of the main reasons that NoSQL databases like Firestore are so popular is that we have the flexibility to optimize for read performance. What we're going to do next is set up a Cloud Function that listens to the donation collection. And every time a new document is written there, it will update an aggregated document that contains the total count, the total amount, and also the last 5 donations that were left.
databases
data management
So this means we could have a 1,000,000 documents in our donations use collection and have all the information needed for the front end UI with a single document read.
The actual data payload will now be much smaller so our front end UI is going to be way more performant for 99% of users. And basically, what we're doing is trade in 20,000 client side document reads for an extra write and an extra Cloud Function invocation. In this case, that's an excellent trade off and it'll make the pricing go from 30 k down to pretty close to the seen go from 30 k down to pretty close to the free tier.
trade
commerce
shipping
purchasing
revenue
One of the drawbacks is that it makes writes to the UI a little bit slower because we have to wait for that Cloud Function to finish before the document is updated. But in this case, it makes a ton of sense because the vast majority of users are reading data and not changing it. I have initialized Firebase Cloud Functions in this project, and I'm also using the new Node 8 runtime which allows me to use async await in just a plain JavaScript function. Will bring in the admin database and then will want to listen to every new donation document that is created in that Firestore collection. Want a new document is created, we'll just want to update the aggregated data for that collection.
So we're no longer displaying the actual donation documents, but instead were displaying the aggregated document based on that collection. There is no best practice for this. But, generally, what I do is I save a document in a collection called aggregation and the ID for that document matches the collection that we're making calculations on. And you really just wanna save the bare minimum amount information you need to make calculations in the UI. In our case, we just need to show the average and the total. So we can calculate these by taking the current aggregated total and then add it to the amount of the new donation, and then we can take the total count and just add 1 to it. And then we'll also save the last 5 donations so we can show those in the view as well.
data
accounting
statistics
documentation
revenue
By saving the last 5 donations here, we can show those in the view, and will only have to execute 1 document read instead of 5, which can make a pretty big difference if you have 2,000,000 users.
And then we'll complete our function by was returning a set operation from it, and we're good to go. Our front end UI looks exactly like it did before, but instead, we're just doing a single document read instead of, say, 20,000. The nice thing about Firestore is that it's easy to anticipate your pricing. Let's imagine that every user reads about 10 documents per visit. That would be about 20,000,000 document reads per 2,000,000 users or about $25 per 2,000,000 merchants or savings of about 99.9% from the startup's initial query setup. I'm gonna go ahead and wrap things up there. If if this video helped you, please like and subscribe. And if you want to learn how to get a quality app shipped to production, consider becoming a pro member at angularfirebase.com. You'll get a free copy of my book, access to my courses, and a whole bunch of other free stuff. Thanks for watching and I'll talk to you soon.
books
youtube
customers
purchasing
commerce
billing
sales
software
Imagine if you woke up one day and your Firebase bill went from 0 to 30,000 US dollars. Well, that's exactly what happened to a startup in Colombia that scaled up to about 2,000,000 daily active users. They really just made one seemingly minor mistake. And with this video, I wanna show you how we can take their $30,000 bill and reduce it down to about $25. If you're new to the channel, make sure to like and subscribe, and you can grab the source code for this project at angularfirebase.com. So you may have already seen this article because it kind of went viral last week on Hacker Noon. But the source of this high bill was because they managed to make over 40,000,000,000 requests to Firestore in less than went 48 hours. So how is that even possible when you only have 2,000,000 users? That means every user would have to make about 20 was in Document Reads. The mistake they made is that they needed to calculate the total for every crowdfunding campaign, but they were doing this buy reading every single document in a collection for every user on the site. So as they got more users and more payments, the problem just compounded to result in a load of document reads. Not only is that super expensive, but it also results in a lot of latency on the client and that's actually how they discovered the problem in the first and I should also point out that this type of mistake is very common and you really won't notice it until you start scaling up end users and you run into these problems with billing and performance. Minutes. I'm gonna show you how to avoid this with strategies both client side and server side. But the number one thing you should do right off the bat is set up a billing alert on Google Cloud Platform. Every Firebase perform. Every Firebase project is tied to a GCP project. So you just have to go into the billing panel and then set up the billing thresholds that you wanna be alerted on. So that way you can find out early if you have some kind of issue that's resulting in billing spikes. But that doesn't really solve the problem at hand because the problem is in the code. So I set up a demo here that does something similar. Basically, it calculates the total amount of donations, the average amount of each donation, and this is all based on a Firestore collection. Should. But I have this app super poorly optimized where I'm reading every document in the collection 3 separate times, one for the total, one for the average, and then one to show the list of donations. So if this collection has 100 documents in it, it will take 300 document reads to present the UI that you see on the left. Believe it or not, this is pretty easy to do and it still performs great at this level so you really wouldn't notice an issue at this point. The startup in the article was using Angular so we'll be doing the same here. Will be using Firestore with AngularFire 2 and then I'm importing Faker so I can generate some fake data for the UI. Now the first thing I want to show you is how we can optimize things client side and I think that's really where you should focus first. So we have 3 different observables hear 1 for the donations, 1 for the total amount, and 1 for the average amount. So the mistake that I'm making here first is that I'm creating a separate observable for the same collection went to show all 3 of these different amounts. That means that we're going to subscribe to 3 different observables, fire off 3 separate read operations to Firestore even though they all depend on the same source data. I'm not really going to explain this code in detail, but basically what we're doing is taking the Firestore collection, which is an array, and then we reduce it down to the total amount of all of the donations in that collection. And then we do the same thing for the average with just a slightly different calculation. And then we subscribe to all 3 of these observables in the HTML using the async pipe. So the main problem here is that we have 3 different data sources that are being created even though we could really get this all done with a single data source. So let's take a look at how we might use use JS to reduce our document reads down from about 300 to 100. What we wanna do is have one observable that can be shared with the other observables that make the calculation for the total and the average. So what I'm going to do first is pipe in the tap operator so we can see when a subscription is created to this observable. Then we'll also console log the number of documents to get an approximate amount of the reads that were executed. Did. So now instead of having the total and the average create their own observables, we'll just go ahead and use the donations observable that we created above. So let's take a look and see if that actually solved the problem in the UI. We should only get one console log. But if we look closely here, we're still getting 3 side effects which should mean that we're sending off 3 sets of reads to Firestore. Luckily, there's an easy way to fix this with RxJS and that's just by piping in the share get a replay operator which will make this a hot observable and allow any other subscribers to always get the last cached value. So now we can be a 100% confident that our code is only sending off one document read to the back end. So that can be a super useful technique, and you can also use an Angular service to inject the same data source into multiple components. But the actual problem that was faced by the start is that we need all of the information from every document in that collection. And to do that client side means that we would have to read every single document every time which is just not feasible. Want, but we can implement a server side solution to make this way more efficient and we can do it with Firebase Cloud Functions. An app like this is very heavy on document reads but were very light on document rights. For example, we might have a 100 people visit the page, but only one of those people actually make a donation and write something to the database. One of the main reasons that NoSQL databases like Firestore are so popular is that we have the flexibility to optimize for read performance. What we're going to do next is set up a Cloud Function that listens to the donation collection. And every time a new document is written there, it will update an aggregated document that contains the total count, the total amount, and also the last 5 donations that were left. So this means we could have a 1,000,000 documents in our donations use collection and have all the information needed for the front end UI with a single document read. The actual data payload will now be much smaller so our front end UI is going to be way more performant for 99% of users. And basically, what we're doing is trade in 20,000 client side document reads for an extra write and an extra Cloud Function invocation. In this case, that's an excellent trade off and it'll make the pricing go from 30 k down to pretty close to the seen go from 30 k down to pretty close to the free tier. One of the drawbacks is that it makes writes to the UI a little bit slower because we have to wait for that Cloud Function to finish before the document is updated. But in this case, it makes a ton of sense because the vast majority of users are reading data and not changing it. I have initialized Firebase Cloud Functions in this project, and I'm also using the new Node 8 runtime which allows me to use async await in just a plain JavaScript function. Will bring in the admin database and then will want to listen to every new donation document that is created in that Firestore collection. Want a new document is created, we'll just want to update the aggregated data for that collection. So we're no longer displaying the actual donation documents, but instead were displaying the aggregated document based on that collection. There is no best practice for this. But, generally, what I do is I save a document in a collection called aggregation and the ID for that document matches the collection that we're making calculations on. And you really just wanna save the bare minimum amount information you need to make calculations in the UI. In our case, we just need to show the average and the total. So we can calculate these by taking the current aggregated total and then add it to the amount of the new donation, and then we can take the total count and just add 1 to it. And then we'll also save the last 5 donations so we can show those in the view as well. By saving the last 5 donations here, we can show those in the view, and will only have to execute 1 document read instead of 5, which can make a pretty big difference if you have 2,000,000 users. And then we'll complete our function by was returning a set operation from it, and we're good to go. Our front end UI looks exactly like it did before, but instead, we're just doing a single document read instead of, say, 20,000. The nice thing about Firestore is that it's easy to anticipate your pricing. Let's imagine that every user reads about 10 documents per visit. That would be about 20,000,000 document reads per 2,000,000 users or about $25 per 2,000,000 merchants or savings of about 99.9% from the startup's initial query setup. I'm gonna go ahead and wrap things up there. If if this video helped you, please like and subscribe. And if you want to learn how to get a quality app shipped to production, consider becoming a pro member at angularfirebase.com. You'll get a free copy of my book, access to my courses, and a whole bunch of other free stuff. Thanks for watching and I'll talk to you soon.
Imagine
imagine
if
if
you
you
woke
woke
up
up
one
one
day
day
and
and
your
your
Firebase
firebase
bill
bill
went
went
from
from
0
0
to
to
30,000
30,000
US
us
dollars.
dollars
Well,
well
that's
that's
exactly
exactly
what
what
happened
happened
to
to
a
a
startup
startup
in
in
Colombia
colombia
that
that
scaled
scaled
up
up
to
to
about
about
2,000,000
2,000,000
daily
daily
active
active
users.
users
They
they
really
really
just
just
made
made
one
one
seemingly
seemingly
minor
minor
mistake.
mistake
And
and
with
with
this
this
video,
video
I
i
wanna
wanna
show
show
you
you
how
how
we
we
can
can
take
take
their
their
$30,000
$30,000
bill
bill
and
and
reduce
reduce
it
it
down
down
to
to
about
about
$25.
$25
If
if
you're
you're
new
new
to
to
the
the
channel,
channel
make
make
sure
sure
to
to
like
like
and
and
subscribe,
subscribe
and
and
you
you
can
can
grab
grab
the
the
source
source
code
code
for
for
this
this
project
project
at
at
angularfirebase.com.
angularfirebase.com
So
so
you
you
may
may
have
have
already
already
seen
seen
this
this
article
article
because
because
it
it
kind
kind
of
of
went
went
viral
viral
last
last
week
week
on
on
Hacker
hacker
Noon.
noon
But
but
the
the
source
source
of
of
this
this
high
high
bill
bill
was
was
because
because
they
they
managed
managed
to
to
make
make
over
over
40,000,000,000
40,000,000,000
requests
requests
to
to
Firestore
firestore
in
in
less
less
than
than
went
went
48
48
hours.
hours
So
so
how
how
is
is
that
that
even
even
possible
possible
when
when
you
you
only
only
have
have
2,000,000
2,000,000
users?
users
That
that
means
means
every
every
user
user
would
would
have
have
to
to
make
make
about
about
20
20
was
was
in
in
Document
document
Reads.
reads
The
the
mistake
mistake
they
they
made
made
is
is
that
that
they
they
needed
needed
to
to
calculate
calculate
the
the
total
total
for
for
every
every
crowdfunding
crowdfunding
campaign,
campaign
but
but
they
they
were
were
doing
doing
this
this
buy
buy
reading
reading
every
every
single
single
document
document
in
in
a
a
collection
collection
for
for
every
every
user
user
on
on
the
the
site.
site
So
so
as
as
they
they
got
got
more
more
users
users
and
and
more
more
payments,
payments
the
the
problem
problem
just
just
compounded
compounded
to
to
result
result
in
in
a
a
load
load
of
of
document
document
reads.
reads
Not
not
only
only
is
is
that
that
super
super
expensive,
expensive
but
but
it
it
also
also
results
results
in
in
a
a
lot
lot
of
of
latency
latency
on
on
the
the
client
client
and
and
that's
that's
actually
actually
how
how
they
they
discovered
discovered
the
the
problem
problem
in
in
the
the
first
first
and
and
I
i
should
should
also
also
point
point
out
out
that
that
this
this
type
type
of
of
mistake
mistake
is
is
very
very
common
common
and
and
you
you
really
really
won't
won't
notice
notice
it
it
until
until
you
you
start
start
scaling
scaling
up
up
end
end
users
users
and
and
you
you
run
run
into
into
these
these
problems
problems
with
with
billing
billing
and
and
performance.
performance
Minutes.
minutes
I'm
i'm
gonna
gonna
show
show
you
you
how
how
to
to
avoid
avoid
this
this
with
with
strategies
strategies
both
both
client
client
side
side
and
and
server
server
side.
side
But
but
the
the
number
number
one
one
thing
thing
you
you
should
should
do
do
right
right
off
off
the
the
bat
bat
is
is
set
set
up
up
a
a
billing
billing
alert
alert
on
on
Google
google
Cloud
cloud
Platform.
platform
Every
every
Firebase
firebase
perform.
perform
Every
every
Firebase
firebase
project
project
is
is
tied
tied
to
to
a
a
GCP
gcp
project.
project
So
so
you
you
just
just
have
have
to
to
go
go
into
into
the
the
billing
billing
panel
panel
and
and
then
then
set
set
up
up
the
the
billing
billing
thresholds
thresholds
that
that
you
you
wanna
wanna
be
be
alerted
alerted
on.
on
So
so
that
that
way
way
you
you
can
can
find
find
out
out
early
early
if
if
you
you
have
have
some
some
kind
kind
of
of
issue
issue
that's
that's
resulting
resulting
in
in
billing
billing
spikes.
spikes
But
but
that
that
doesn't
doesn't
really
really
solve
solve
the
the
problem
problem
at
at
hand
hand
because
because
the
the
problem
problem
is
is
in
in
the
the
code.
code
So
so
I
i
set
set
up
up
a
a
demo
demo
here
here
that
that
does
does
something
something
similar.
similar
Basically,
basically
it
it
calculates
calculates
the
the
total
total
amount
amount
of
of
donations,
donations
the
the
average
average
amount
amount
of
of
each
each
donation,
donation
and
and
this
this
is
is
all
all
based
based
on
on
a
a
Firestore
firestore
collection.
collection
Should.
should
But
but
I
i
have
have
this
this
app
app
super
super
poorly
poorly
optimized
optimized
where
where
I'm
i'm
reading
reading
every
every
document
document
in
in
the
the
collection
collection
3
3
separate
separate
times,
times
one
one
for
for
the
the
total,
total
one
one
for
for
the
the
average,
average
and
and
then
then
one
one
to
to
show
show
the
the
list
list
of
of
donations.
donations
So
so
if
if
this
this
collection
collection
has
has
100
100
documents
documents
in
in
it,
it
it
it
will
will
take
take
300
300
document
document
reads
reads
to
to
present
present
the
the
UI
ui
that
that
you
you
see
see
on
on
the
the
left.
left
Believe
believe
it
it
or
or
not,
not
this
this
is
is
pretty
pretty
easy
easy
to
to
do
do
and
and
it
it
still
still
performs
performs
great
great
at
at
this
this
level
level
so
so
you
you
really
really
wouldn't
wouldn't
notice
notice
an
an
issue
issue
at
at
this
this
point.
point
The
the
startup
startup
in
in
the
the
article
article
was
was
using
using
Angular
angular
so
so
we'll
we'll
be
be
doing
doing
the
the
same
same
here.
here
Will
will
be
be
using
using
Firestore
firestore
with
with
AngularFire
angularfire
2
2
and
and
then
then
I'm
i'm
importing
importing
Faker
faker
so
so
I
i
can
can
generate
generate
some
some
fake
fake
data
data
for
for
the
the
UI.
ui
Now
now
the
the
first
first
thing
thing
I
i
want
want
to
to
show
show
you
you
is
is
how
how
we
we
can
can
optimize
optimize
things
things
client
client
side
side
and
and
I
i
think
think
that's
that's
really
really
where
where
you
you
should
should
focus
focus
first.
first
So
so
we
we
have
have
3
3
different
different
observables
observables
hear
hear
1
1
for
for
the
the
donations,
donations
1
1
for
for
the
the
total
total
amount,
amount
and
and
1
1
for
for
the
the
average
average
amount.
amount
So
so
the
the
mistake
mistake
that
that
I'm
i'm
making
making
here
here
first
first
is
is
that
that
I'm
i'm
creating
creating
a
a
separate
separate
observable
observable
for
for
the
the
same
same
collection
collection
went
went
to
to
show
show
all
all
3
3
of
of
these
these
different
different
amounts.
amounts
That
that
means
means
that
that
we're
we're
going
going
to
to
subscribe
subscribe
to
to
3
3
different
different
observables,
observables
fire
fire
off
off
3
3
separate
separate
read
read
operations
operations
to
to
Firestore
firestore
even
even
though
though
they
they
all
all
depend
depend
on
on
the
the
same
same
source
source
data.
data
I'm
i'm
not
not
really
really
going
going
to
to
explain
explain
this
this
code
code
in
in
detail,
detail
but
but
basically
basically
what
what
we're
we're
doing
doing
is
is
taking
taking
the
the
Firestore
firestore
collection,
collection
which
which
is
is
an
an
array,
array
and
and
then
then
we
we
reduce
reduce
it
it
down
down
to
to
the
the
total
total
amount
amount
of
of
all
all
of
of
the
the
donations
donations
in
in
that
that
collection.
collection
And
and
then
then
we
we
do
do
the
the
same
same
thing
thing
for
for
the
the
average
average
with
with
just
just
a
a
slightly
slightly
different
different
calculation.
calculation
And
and
then
then
we
we
subscribe
subscribe
to
to
all
all
3
3
of
of
these
these
observables
observables
in
in
the
the
HTML
html
using
using
the
the
async
async
pipe.
pipe
So
so
the
the
main
main
problem
problem
here
here
is
is
that
that
we
we
have
have
3
3
different
different
data
data
sources
sources
that
that
are
are
being
being
created
created
even
even
though
though
we
we
could
could
really
really
get
get
this
this
all
all
done
done
with
with
a
a
single
single
data
data
source.
source
So
so
let's
let's
take
take
a
a
look
look
at
at
how
how
we
we
might
might
use
use
use
use
JS
js
to
to
reduce
reduce
our
our
document
document
reads
reads
down
down
from
from
about
about
300
300
to
to
100.
100
What
what
we
we
wanna
wanna
do
do
is
is
have
have
one
one
observable
observable
that
that
can
can
be
be
shared
shared
with
with
the
the
other
other
observables
observables
that
that
make
make
the
the
calculation
calculation
for
for
the
the
total
total
and
and
the
the
average.
average
So
so
what
what
I'm
i'm
going
going
to
to
do
do
first
first
is
is
pipe
pipe
in
in
the
the
tap
tap
operator
operator
so
so
we
we
can
can
see
see
when
when
a
a
subscription
subscription
is
is
created
created
to
to
this
this
observable.
observable
Then
then
we'll
we'll
also
also
console
console
log
log
the
the
number
number
of
of
documents
documents
to
to
get
get
an
an
approximate
approximate
amount
amount
of
of
the
the
reads
reads
that
that
were
were
executed.
executed
Did.
did
So
so
now
now
instead
instead
of
of
having
having
the
the
total
total
and
and
the
the
average
average
create
create
their
their
own
own
observables,
observables
we'll
we'll
just
just
go
go
ahead
ahead
and
and
use
use
the
the
donations
donations
observable
observable
that
that
we
we
created
created
above.
above
So
so
let's
let's
take
take
a
a
look
look
and
and
see
see
if
if
that
that
actually
actually
solved
solved
the
the
problem
problem
in
in
the
the
UI.
ui
We
we
should
should
only
only
get
get
one
one
console
console
log.
log
But
but
if
if
we
we
look
look
closely
closely
here,
here
we're
we're
still
still
getting
getting
3
3
side
side
effects
effects
which
which
should
should
mean
mean
that
that
we're
we're
sending
sending
off
off
3
3
sets
sets
of
of
reads
reads
to
to
Firestore.
firestore
Luckily,
luckily
there's
there's
an
an
easy
easy
way
way
to
to
fix
fix
this
this
with
with
RxJS
rxjs
and
and
that's
that's
just
just
by
by
piping
piping
in
in
the
the
share
share
get
get
a
a
replay
replay
operator
operator
which
which
will
will
make
make
this
this
a
a
hot
hot
observable
observable
and
and
allow
allow
any
any
other
other
subscribers
subscribers
to
to
always
always
get
get
the
the
last
last
cached
cached
value.
value
So
so
now
now
we
we
can
can
be
be
a
a
100%
100%
confident
confident
that
that
our
our
code
code
is
is
only
only
sending
sending
off
off
one
one
document
document
read
read
to
to
the
the
back
back
end.
end
So
so
that
that
can
can
be
be
a
a
super
super
useful
useful
technique,
technique
and
and
you
you
can
can
also
also
use
use
an
an
Angular
angular
service
service
to
to
inject
inject
the
the
same
same
data
data
source
source
into
into
multiple
multiple
components.
components
But
but
the
the
actual
actual
problem
problem
that
that
was
was
faced
faced
by
by
the
the
start
start
is
is
that
that
we
we
need
need
all
all
of
of
the
the
information
information
from
from
every
every
document
document
in
in
that
that
collection.
collection
And
and
to
to
do
do
that
that
client
client
side
side
means
means
that
that
we
we
would
would
have
have
to
to
read
read
every
every
single
single
document
document
every
every
time
time
which
which
is
is
just
just
not
not
feasible.
feasible
Want,
want
but
but
we
we
can
can
implement
implement
a
a
server
server
side
side
solution
solution
to
to
make
make
this
this
way
way
more
more
efficient
efficient
and
and
we
we
can
can
do
do
it
it
with
with
Firebase
firebase
Cloud
cloud
Functions.
functions
An
an
app
app
like
like
this
this
is
is
very
very
heavy
heavy
on
on
document
document
reads
reads
but
but
were
were
very
very
light
light
on
on
document
document
rights.
rights
For
for
example,
example
we
we
might
might
have
have
a
a
100
100
people
people
visit
visit
the
the
page,
page
but
but
only
only
one
one
of
of
those
those
people
people
actually
actually
make
make
a
a
donation
donation
and
and
write
write
something
something
to
to
the
the
database.
database
One
one
of
of
the
the
main
main
reasons
reasons
that
that
NoSQL
nosql
databases
databases
like
like
Firestore
firestore
are
are
so
so
popular
popular
is
is
that
that
we
we
have
have
the
the
flexibility
flexibility
to
to
optimize
optimize
for
for
read
read
performance.
performance
What
what
we're
we're
going
going
to
to
do
do
next
next
is
is
set
set
up
up
a
a
Cloud
cloud
Function
function
that
that
listens
listens
to
to
the
the
donation
donation
collection.
collection
And
and
every
every
time
time
a
a
new
new
document
document
is
is
written
written
there,
there
it
it
will
will
update
update
an
an
aggregated
aggregated
document
document
that
that
contains
contains
the
the
total
total
count,
count
the
the
total
total
amount,
amount
and
and
also
also
the
the
last
last
5
5
donations
donations
that
that
were
were
left.
left
So
so
this
this
means
means
we
we
could
could
have
have
a
a
1,000,000
1,000,000
documents
documents
in
in
our
our
donations
donations
use
use
collection
collection
and
and
have
have
all
all
the
the
information
information
needed
needed
for
for
the
the
front
front
end
end
UI
ui
with
with
a
a
single
single
document
document
read.
read
The
the
actual
actual
data
data
payload
payload
will
will
now
now
be
be
much
much
smaller
smaller
so
so
our
our
front
front
end
end
UI
ui
is
is
going
going
to
to
be
be
way
way
more
more
performant
performant
for
for
99%
99%
of
of
users.
users
And
and
basically,
basically
what
what
we're
we're
doing
doing
is
is
trade
trade
in
in
20,000
20,000
client
client
side
side
document
document
reads
reads
for
for
an
an
extra
extra
write
write
and
and
an
an
extra
extra
Cloud
cloud
Function
function
invocation.
invocation
In
in
this
this
case,
case
that's
that's
an
an
excellent
excellent
trade
trade
off
off
and
and
it'll
it'll
make
make
the
the
pricing
pricing
go
go
from
from
30
30
k
k
down
down
to
to
pretty
pretty
close
close
to
to
the
the
seen
seen
go
go
from
from
30
30
k
k
down
down
to
to
pretty
pretty
close
close
to
to
the
the
free
free
tier.
tier
One
one
of
of
the
the
drawbacks
drawbacks
is
is
that
that
it
it
makes
makes
writes
writes
to
to
the
the
UI
ui
a
a
little
little
bit
bit
slower
slower
because
because
we
we
have
have
to
to
wait
wait
for
for
that
that
Cloud
cloud
Function
function
to
to
finish
finish
before
before
the
the
document
document
is
is
updated.
updated
But
but
in
in
this
this
case,
case
it
it
makes
makes
a
a
ton
ton
of
of
sense
sense
because
because
the
the
vast
vast
majority
majority
of
of
users
users
are
are
reading
reading
data
data
and
and
not
not
changing
changing
it.
it
I
i
have
have
initialized
initialized
Firebase
firebase
Cloud
cloud
Functions
functions
in
in
this
this
project,
project
and
and
I'm
i'm
also
also
using
using
the
the
new
new
Node
node
8
8
runtime
runtime
which
which
allows
allows
me
me
to
to
use
use
async
async
await
await
in
in
just
just
a
a
plain
plain
JavaScript
javascript
function.
function
Will
will
bring
bring
in
in
the
the
admin
admin
database
database
and
and
then
then
will
will
want
want
to
to
listen
listen
to
to
every
every
new
new
donation
donation
document
document
that
that
is
is
created
created
in
in
that
that
Firestore
firestore
collection.
collection
Want
want
a
a
new
new
document
document
is
is
created,
created
we'll
we'll
just
just
want
want
to
to
update
update
the
the
aggregated
aggregated
data
data
for
for
that
that
collection.
collection
So
so
we're
we're
no
no
longer
longer
displaying
displaying
the
the
actual
actual
donation
donation
documents,
documents
but
but
instead
instead
were
were
displaying
displaying
the
the
aggregated
aggregated
document
document
based
based
on
on
that
that
collection.
collection
There
there
is
is
no
no
best
best
practice
practice
for
for
this.
this
But,
but
generally,
generally
what
what
I
i
do
do
is
is
I
i
save
save
a
a
document
document
in
in
a
a
collection
collection
called
called
aggregation
aggregation
and
and
the
the
ID
id
for
for
that
that
document
document
matches
matches
the
the
collection
collection
that
that
we're
we're
making
making
calculations
calculations
on.
on
And
and
you
you
really
really
just
just
wanna
wanna
save
save
the
the
bare
bare
minimum
minimum
amount
amount
information
information
you
you
need
need
to
to
make
make
calculations
calculations
in
in
the
the
UI.
ui
In
in
our
our
case,
case
we
we
just
just
need
need
to
to
show
show
the
the
average
average
and
and
the
the
total.
total
So
so
we
we
can
can
calculate
calculate
these
these
by
by
taking
taking
the
the
current
current
aggregated
aggregated
total
total
and
and
then
then
add
add
it
it
to
to
the
the
amount
amount
of
of
the
the
new
new
donation,
donation
and
and
then
then
we
we
can
can
take
take
the
the
total
total
count
count
and
and
just
just
add
add
1
1
to
to
it.
it
And
and
then
then
we'll
we'll
also
also
save
save
the
the
last
last
5
5
donations
donations
so
so
we
we
can
can
show
show
those
those
in
in
the
the
view
view
as
as
well.
well
By
by
saving
saving
the
the
last
last
5
5
donations
donations
here,
here
we
we
can
can
show
show
those
those
in
in
the
the
view,
view
and
and
will
will
only
only
have
have
to
to
execute
execute
1
1
document
document
read
read
instead
instead
of
of
5,
5
which
which
can
can
make
make
a
a
pretty
pretty
big
big
difference
difference
if
if
you
you
have
have
2,000,000
2,000,000
users.
users
And
and
then
then
we'll
we'll
complete
complete
our
our
function
function
by
by
was
was
returning
returning
a
a
set
set
operation
operation
from
from
it,
it
and
and
we're
we're
good
good
to
to
go.
go
Our
our
front
front
end
end
UI
ui
looks
looks
exactly
exactly
like
like
it
it
did
did
before,
before
but
but
instead,
instead
we're
we're
just
just
doing
doing
a
a
single
single
document
document
read
read
instead
instead
of,
of
say,
say
20,000.
20,000
The
the
nice
nice
thing
thing
about
about
Firestore
firestore
is
is
that
that
it's
it's
easy
easy
to
to
anticipate
anticipate
your
your
pricing.
pricing
Let's
let's
imagine
imagine
that
that
every
every
user
user
reads
reads
about
about
10
10
documents
documents
per
per
visit.
visit
That
that
would
would
be
be
about
about
20,000,000
20,000,000
document
document
reads
reads
per
per
2,000,000
2,000,000
users
users
or
or
about
about
$25
$25
per
per
2,000,000
2,000,000
merchants
merchants
or
or
savings
savings
of
of
about
about
99.9%
99.9%
from
from
the
the
startup's
startup's
initial
initial
query
query
setup.
setup
I'm
i'm
gonna
gonna
go
go
ahead
ahead
and
and
wrap
wrap
things
things
up
up
there.
there
If
if
if
if
this
this
video
video
helped
helped
you,
you
please
please
like
like
and
and
subscribe.
subscribe
And
and
if
if
you
you
want
want
to
to
learn
learn
how
how
to
to
get
get
a
a
quality
quality
app
app
shipped
shipped
to
to
production,
production
consider
consider
becoming
becoming
a
a
pro
pro
member
member
at
at
angularfirebase.com.
angularfirebase.com
You'll
you'll
get
get
a
a
free
free
copy
copy
of
of
my
my
book,
book
access
access
to
to
my
my
courses,
courses
and
and
a
a
whole
whole
bunch
bunch
of
of
other
other
free
free
stuff.
stuff
Thanks
thanks
for
for
watching
watching
and
and
I'll
i'll
talk
talk
to
to
you
you
soon.
soon
success
The speaker discusses the issue of over-treatment in a startup in Colombia, which has caused a high bill of $25,000 per user. They recommend setting up a billing alert on Google Cloud Platform and using the Fir analysis library to optimize things client side. The speaker also explains how to use the variables in a single data source to reduce document reads and how to use the billing alert and contribution of each call. They suggest using a Cloud function to optimize front end UI and trade in 20,000 document reads for an extra write and an extra Cloud function. The speaker also discusses scalability and front end UI with Fir analysis, and suggests implementing a server side solution and using an app called denies.
How to NOT get a 30K Firebase Bill
Imagine if you woke up one day and your Firebase bill went from 0 to 30,000 US dollars. Well, that's exactly what happened to a startup in Colombia that scaled up to about 2,000,000 daily active users. They really just made one seemingly minor mistake. And with this video, I wanna show you how we can take their $30,000 bill and reduce it down to about $25. If you're new to the channel, make sure to like and subscribe, and you can grab the source code for this project at angularfirebase.com. So you may have already seen this article because it kind of went viral last week on Hacker Noon. But the source of this high bill was because they managed to make over 40,000,000,000 requests to Firestore in less than went 48 hours. So how is that even possible when you only have 2,000,000 users? That means every user would have to make about 20 was in Document Reads. The mistake they made is that they needed to calculate the total for every crowdfunding campaign, but they were doing this buy reading every single document in a collection for every user on the site. So as they got more users and more payments, the problem just compounded to result in a load of document reads. Not only is that super expensive, but it also results in a lot of latency on the client and that's actually how they discovered the problem in the first and I should also point out that this type of mistake is very common and you really won't notice it until you start scaling up end users and you run into these problems with billing and performance. Minutes. I'm gonna show you how to avoid this with strategies both client side and server side. But the number one thing you should do right off the bat is set up a billing alert on Google Cloud Platform. Every Firebase perform. Every Firebase project is tied to a GCP project. So you just have to go into the billing panel and then set up the billing thresholds that you wanna be alerted on. So that way you can find out early if you have some kind of issue that's resulting in billing spikes. But that doesn't really solve the problem at hand because the problem is in the code. So I set up a demo here that does something similar. Basically, it calculates the total amount of donations, the average amount of each donation, and this is all based on a Firestore collection. Should. But I have this app super poorly optimized where I'm reading every document in the collection 3 separate times, one for the total, one for the average, and then one to show the list of donations. So if this collection has 100 documents in it, it will take 300 document reads to present the UI that you see on the left. Believe it or not, this is pretty easy to do and it still performs great at this level so you really wouldn't notice an issue at this point. The startup in the article was using Angular so we'll be doing the same here. Will be using Firestore with AngularFire 2 and then I'm importing Faker so I can generate some fake data for the UI. Now the first thing I want to show you is how we can optimize things client side and I think that's really where you should focus first. So we have 3 different observables hear 1 for the donations, 1 for the total amount, and 1 for the average amount. So the mistake that I'm making here first is that I'm creating a separate observable for the same collection went to show all 3 of these different amounts. That means that we're going to subscribe to 3 different observables, fire off 3 separate read operations to Firestore even though they all depend on the same source data. I'm not really going to explain this code in detail, but basically what we're doing is taking the Firestore collection, which is an array, and then we reduce it down to the total amount of all of the donations in that collection. And then we do the same thing for the average with just a slightly different calculation. And then we subscribe to all 3 of these observables in the HTML using the async pipe. So the main problem here is that we have 3 different data sources that are being created even though we could really get this all done with a single data source. So let's take a look at how we might use use JS to reduce our document reads down from about 300 to 100. What we wanna do is have one observable that can be shared with the other observables that make the calculation for the total and the average. So what I'm going to do first is pipe in the tap operator so we can see when a subscription is created to this observable. Then we'll also console log the number of documents to get an approximate amount of the reads that were executed. Did. So now instead of having the total and the average create their own observables, we'll just go ahead and use the donations observable that we created above. So let's take a look and see if that actually solved the problem in the UI. We should only get one console log. But if we look closely here, we're still getting 3 side effects which should mean that we're sending off 3 sets of reads to Firestore. Luckily, there's an easy way to fix this with RxJS and that's just by piping in the share get a replay operator which will make this a hot observable and allow any other subscribers to always get the last cached value. So now we can be a 100% confident that our code is only sending off one document read to the back end. So that can be a super useful technique, and you can also use an Angular service to inject the same data source into multiple components. But the actual problem that was faced by the start is that we need all of the information from every document in that collection. And to do that client side means that we would have to read every single document every time which is just not feasible. Want, but we can implement a server side solution to make this way more efficient and we can do it with Firebase Cloud Functions. An app like this is very heavy on document reads but were very light on document rights. For example, we might have a 100 people visit the page, but only one of those people actually make a donation and write something to the database. One of the main reasons that NoSQL databases like Firestore are so popular is that we have the flexibility to optimize for read performance. What we're going to do next is set up a Cloud Function that listens to the donation collection. And every time a new document is written there, it will update an aggregated document that contains the total count, the total amount, and also the last 5 donations that were left. So this means we could have a 1,000,000 documents in our donations use collection and have all the information needed for the front end UI with a single document read. The actual data payload will now be much smaller so our front end UI is going to be way more performant for 99% of users. And basically, what we're doing is trade in 20,000 client side document reads for an extra write and an extra Cloud Function invocation. In this case, that's an excellent trade off and it'll make the pricing go from 30 k down to pretty close to the seen go from 30 k down to pretty close to the free tier. One of the drawbacks is that it makes writes to the UI a little bit slower because we have to wait for that Cloud Function to finish before the document is updated. But in this case, it makes a ton of sense because the vast majority of users are reading data and not changing it. I have initialized Firebase Cloud Functions in this project, and I'm also using the new Node 8 runtime which allows me to use async await in just a plain JavaScript function. Will bring in the admin database and then will want to listen to every new donation document that is created in that Firestore collection. Want a new document is created, we'll just want to update the aggregated data for that collection. So we're no longer displaying the actual donation documents, but instead were displaying the aggregated document based on that collection. There is no best practice for this. But, generally, what I do is I save a document in a collection called aggregation and the ID for that document matches the collection that we're making calculations on. And you really just wanna save the bare minimum amount information you need to make calculations in the UI. In our case, we just need to show the average and the total. So we can calculate these by taking the current aggregated total and then add it to the amount of the new donation, and then we can take the total count and just add 1 to it. And then we'll also save the last 5 donations so we can show those in the view as well. By saving the last 5 donations here, we can show those in the view, and will only have to execute 1 document read instead of 5, which can make a pretty big difference if you have 2,000,000 users. And then we'll complete our function by was returning a set operation from it, and we're good to go. Our front end UI looks exactly like it did before, but instead, we're just doing a single document read instead of, say, 20,000. The nice thing about Firestore is that it's easy to anticipate your pricing. Let's imagine that every user reads about 10 documents per visit. That would be about 20,000,000 document reads per 2,000,000 users or about $25 per 2,000,000 merchants or savings of about 99.9% from the startup's initial query setup. I'm gonna go ahead and wrap things up there. If if this video helped you, please like and subscribe. And if you want to learn how to get a quality app shipped to production, consider becoming a pro member at angularfirebase.com. You'll get a free copy of my book, access to my courses, and a whole bunch of other free stuff. Thanks for watching and I'll talk to you soon.
The speaker discusses the issue of over-treatment in a startup in Colombia, which has caused a high bill of $25,000 per user. They recommend setting up a billing alert on Google Cloud Platform and using the Fir analysis library to optimize things client side. The speaker also explains how to use the variables in a single data source to reduce document reads and how to use the billing alert and contribution of each call. They suggest using a Cloud function to optimize front end UI and trade in 20,000 document reads for an extra write and an extra Cloud function. The speaker also discusses scalability and front end UI with Fir analysis, and suggests implementing a server side solution and using an app called denies.

        [INST]
        You are youtuber called Fireship you make engaging high-intensity and entertaining coding tutorials and tech news. 
        you covers a wide range of topics relevant to programmers, aiming to help them learn and improve their skills quickly.
        
        Given the title of the video : How to NOT get a 30K Firebase Bill 
        and a small summary : The speaker discusses the issue of over-treatment in a startup in Colombia, which has caused a high bill of $25,000 per user. They recommend setting up a billing alert on Google Cloud Platform and using the Fir analysis library to optimize things client side. The speaker also explains how to use the variables in a single data source to reduce document reads and how to use the billing alert and contribution of each call. They suggest using a Cloud function to optimize front end UI and trade in 20,000 document reads for an extra write and an extra Cloud function. The speaker also discusses scalability and front end UI with Fir analysis, and suggests implementing a server side solution and using an app called denies.
        [/INST]
        
        Generate the video : Imagine if you woke up one day and your Firebase bill went from 0 to 30,000 US dollars. Well, that's exactly what happened to a startup in Colombia that scaled up to about 2,000,000 daily active users. They really just made one seemingly minor mistake. And with this video, I wanna show you how we can take their $30,000 bill and reduce it down to about $25. If you're new to the channel, make sure to like and subscribe, and you can grab the source code for this project at angularfirebase.com. So you may have already seen this article because it kind of went viral last week on Hacker Noon. But the source of this high bill was because they managed to make over 40,000,000,000 requests to Firestore in less than went 48 hours. So how is that even possible when you only have 2,000,000 users? That means every user would have to make about 20 was in Document Reads. The mistake they made is that they needed to calculate the total for every crowdfunding campaign, but they were doing this buy reading every single document in a collection for every user on the site. So as they got more users and more payments, the problem just compounded to result in a load of document reads. Not only is that super expensive, but it also results in a lot of latency on the client and that's actually how they discovered the problem in the first and I should also point out that this type of mistake is very common and you really won't notice it until you start scaling up end users and you run into these problems with billing and performance. Minutes. I'm gonna show you how to avoid this with strategies both client side and server side. But the number one thing you should do right off the bat is set up a billing alert on Google Cloud Platform. Every Firebase perform. Every Firebase project is tied to a GCP project. So you just have to go into the billing panel and then set up the billing thresholds that you wanna be alerted on. So that way you can find out early if you have some kind of issue that's resulting in billing spikes. But that doesn't really solve the problem at hand because the problem is in the code. So I set up a demo here that does something similar. Basically, it calculates the total amount of donations, the average amount of each donation, and this is all based on a Firestore collection. Should. But I have this app super poorly optimized where I'm reading every document in the collection 3 separate times, one for the total, one for the average, and then one to show the list of donations. So if this collection has 100 documents in it, it will take 300 document reads to present the UI that you see on the left. Believe it or not, this is pretty easy to do and it still performs great at this level so you really wouldn't notice an issue at this point. The startup in the article was using Angular so we'll be doing the same here. Will be using Firestore with AngularFire 2 and then I'm importing Faker so I can generate some fake data for the UI. Now the first thing I want to show you is how we can optimize things client side and I think that's really where you should focus first. So we have 3 different observables hear 1 for the donations, 1 for the total amount, and 1 for the average amount. So the mistake that I'm making here first is that I'm creating a separate observable for the same collection went to show all 3 of these different amounts. That means that we're going to subscribe to 3 different observables, fire off 3 separate read operations to Firestore even though they all depend on the same source data. I'm not really going to explain this code in detail, but basically what we're doing is taking the Firestore collection, which is an array, and then we reduce it down to the total amount of all of the donations in that collection. And then we do the same thing for the average with just a slightly different calculation. And then we subscribe to all 3 of these observables in the HTML using the async pipe. So the main problem here is that we have 3 different data sources that are being created even though we could really get this all done with a single data source. So let's take a look at how we might use use JS to reduce our document reads down from about 300 to 100. What we wanna do is have one observable that can be shared with the other observables that make the calculation for the total and the average. So what I'm going to do first is pipe in the tap operator so we can see when a subscription is created to this observable. Then we'll also console log the number of documents to get an approximate amount of the reads that were executed. Did. So now instead of having the total and the average create their own observables, we'll just go ahead and use the donations observable that we created above. So let's take a look and see if that actually solved the problem in the UI. We should only get one console log. But if we look closely here, we're still getting 3 side effects which should mean that we're sending off 3 sets of reads to Firestore. Luckily, there's an easy way to fix this with RxJS and that's just by piping in the share get a replay operator which will make this a hot observable and allow any other subscribers to always get the last cached value. So now we can be a 100% confident that our code is only sending off one document read to the back end. So that can be a super useful technique, and you can also use an Angular service to inject the same data source into multiple components. But the actual problem that was faced by the start is that we need all of the information from every document in that collection. And to do that client side means that we would have to read every single document every time which is just not feasible. Want, but we can implement a server side solution to make this way more efficient and we can do it with Firebase Cloud Functions. An app like this is very heavy on document reads but were very light on document rights. For example, we might have a 100 people visit the page, but only one of those people actually make a donation and write something to the database. One of the main reasons that NoSQL databases like Firestore are so popular is that we have the flexibility to optimize for read performance. What we're going to do next is set up a Cloud Function that listens to the donation collection. And every time a new document is written there, it will update an aggregated document that contains the total count, the total amount, and also the last 5 donations that were left. So this means we could have a 1,000,000 documents in our donations use collection and have all the information needed for the front end UI with a single document read. The actual data payload will now be much smaller so our front end UI is going to be way more performant for 99% of users. And basically, what we're doing is trade in 20,000 client side document reads for an extra write and an extra Cloud Function invocation. In this case, that's an excellent trade off and it'll make the pricing go from 30 k down to pretty close to the seen go from 30 k down to pretty close to the free tier. One of the drawbacks is that it makes writes to the UI a little bit slower because we have to wait for that Cloud Function to finish before the document is updated. But in this case, it makes a ton of sense because the vast majority of users are reading data and not changing it. I have initialized Firebase Cloud Functions in this project, and I'm also using the new Node 8 runtime which allows me to use async await in just a plain JavaScript function. Will bring in the admin database and then will want to listen to every new donation document that is created in that Firestore collection. Want a new document is created, we'll just want to update the aggregated data for that collection. So we're no longer displaying the actual donation documents, but instead were displaying the aggregated document based on that collection. There is no best practice for this. But, generally, what I do is I save a document in a collection called aggregation and the ID for that document matches the collection that we're making calculations on. And you really just wanna save the bare minimum amount information you need to make calculations in the UI. In our case, we just need to show the average and the total. So we can calculate these by taking the current aggregated total and then add it to the amount of the new donation, and then we can take the total count and just add 1 to it. And then we'll also save the last 5 donations so we can show those in the view as well. By saving the last 5 donations here, we can show those in the view, and will only have to execute 1 document read instead of 5, which can make a pretty big difference if you have 2,000,000 users. And then we'll complete our function by was returning a set operation from it, and we're good to go. Our front end UI looks exactly like it did before, but instead, we're just doing a single document read instead of, say, 20,000. The nice thing about Firestore is that it's easy to anticipate your pricing. Let's imagine that every user reads about 10 documents per visit. That would be about 20,000,000 document reads per 2,000,000 users or about $25 per 2,000,000 merchants or savings of about 99.9% from the startup's initial query setup. I'm gonna go ahead and wrap things up there. If if this video helped you, please like and subscribe. And if you want to learn how to get a quality app shipped to production, consider becoming a pro member at angularfirebase.com. You'll get a free copy of my book, access to my courses, and a whole bunch of other free stuff. Thanks for watching and I'll talk to you soon.
        