Five useful data modeling techniques in Cloud Firestore 🔥 that you may not know about. Checkout the full course here 👉https://fireship.io/courses/firestore-data-modeling/

- 25% off PRO Membership use code: lORhwXd2
- Cloud Firestore https://firebase.google.com/docs/firestore/
2024-02-04T06:49:42.266Z
nova-2
2-general-nova
2024-01-09.29447
30089e05-99d1-4376-b32e-c263170674af
4df6ae5b-9cac-406b-9677-52c7ee36001a
659871bc90c55439e445e80683301bf3b02a888a09a865257f5b7f8f88e3fae6
67875a7f-c9c4-48a0-aa55-5bdb8a91c34a
deprecated
One of the most challenging aspects of building any app is determining the optimal data structure.
Ideally, you want to maximize performance while minimizing costs.
But that's easier said than done, and especially so with a NoSQL database like Firestore, because you really have to plan ahead on how you model your data.
Last week, I asked everybody on our Slack channel to send me the biggest challenges that they faced modeling data in Firestore, store, and this conversation snowballed into an entire new course on Fireship dedicated completely to NoSQL data modeling and queries.
So if this is a topic you're interested in, consider becoming a Pro Member to get access to that course, and I'll leave a discount code in the video description.
In today's video, we're going to look at 5 different ways you can structure and query your data that you may not know about.
And each technique that we'll look at is based on a real submission from the community.
A minute ago I said that you have to plan ahead when structuring your data.
So what do I mean by that exactly?
Well, the basic idea is that you want to pre render your data so that it fits the view or the screen in your app as closely as possible.
Ideally, you only need to make one document read or one query to a collection to fill the entire view with data.
This differs from a relational SQL database where the idea is to break your data into a lot small pieces, and then join it all together server side.
Now I've heard people say that Firestore or NoSQL databases in general are not good for relational data modeling, but that's actually not true, and it's really just a matter of rethinking the way that you model data in general.
And what you'll find is that you can solve all the same problems that you can with SQL, but you can do it in a way is faster, less expensive, and more flexible.
With that being said, let's go ahead and jump into our first data model and access pattern.
This suggestion comes from Somrat, and he wants to query a collection based on whether or not a user was mentioned in a post in that collection.
This is similar to something like Twitter where if you mention a username the at symbol, it will notify that user that they've been mentioned, and they also might want to see a list of all the tweets that they've been mentioned in.
In this case, we have a many to many relationship where a tweet can have many mentioned users, and a user can be mentioned in many tweets.
But the thing to keep in mind here is that a tweet or a post can maybe only have 1 to 10 mentioned users, but a user can be mentioned in potentially billions of tweets.
In situations like this, it's usually best to have the document with the smaller side of the relational data manage the relationships.
In in this example, we have an array of usernames embedded on the tweet or post document.
You'd implement this logic somewhere in your front end code using regex is to extract the usernames out of the post text and then duplicate them on the same document as an array.
Now it becomes very easy to query all the tweets that a user is mentioned in by making a reference to the post collection, then we can use the array contains operator to get all the tweets that have a corresponding user ID in that mentions array.
Then we could use a cloud function to listen for a new post to be created and then send a push notification or an email to your user when they're mentioned.
So the main limitation what array contains is that you can only query for 1 item at a time.
And that leads me to the next challenge, which is a category system that allows you to filter by multiple categories at the same time while also being able to see if a post contains a given category out of potentially multiple past values and also the ability to filter by not containing a certain value.
So these requirements roughly translate into an and or or not statement in a SQL database.
I'm gonna go ahead and call these tags instead of categories, but the general idea here is the same.
We have a tags collection, which is really just there to hold any extra data about a tag, like a description or a URL or something like that.
And I'd recommend giving your tags a unique ID that's something descriptive that you can actually show in the UI.
And then you'll take that ID and associate it to a post by using a map.
So we have a map called tags, and then each key in that map is the ID from the tags collection.
And the cool thing about Firestore is that it automatically indexes the keys in the map, which means we can query them without any additional configuration.
First, let's take a look at how we can do a logical inquiry, or in other words, we want to get all of the items that have both of these tags present.
We can do this by simply chaining together where statements.
And as long as they're using the equal to operator, we can do this as many times as we want.
So this works great if you're filtering by multiple Boolean values.
But if you throw in a range operator, that means you'll need a composite index.
And keep in mind, you can only use 1 range operator per query, so that's a limitation that you'll want to consider.
Now if you wanted to make something that was more like an or query, you can just make multiple queries for separate tags.
We're able to run these queries concurrently, and then we can just join them together and filter throughout any duplicates in our client side code.
Now the most difficult part of this challenge is implementing the not logic.
The values on the map are set up as booleans, so the most straightforward approach here would be to add all of the tags to every document and then set them to false by default.
This would be perfectly fine if you have a finite set of tags and know their values up front, but it would be much more challenging if tags are generated by users and there could be potentially billions of tags.
So that's a limitation to consider.
But keep in mind that if you have a numeric value or if you have a string value with some sort of ordering embedded in it, then you can simulate a knot query by using range operators.
For example, if we wanted to get all of the items that were not $20, then we could query on either side of that range.
Or in other words, all the items that are less than $20 and all the items that are more than $20.
And if you're doing a lot of stuff like this, it might be time to look into something like Algolia to index some of your data in a full text search engine.
That's what I do for Fireship, and Firebase and Algolia work really well together.
Now moving on to our next data model, this one comes from Lenny Cunningham and his flight now dot net app.
Geolocation queries bring up a really interesting data modeling technique called composite strings.
Not only do composite strings allow us to do things like geohashing, but it also allows us to do things like tree traversals and threaded comments.
Take a look at this tree structure we have here where each letter represents a document in the database.
What I'm going to show you next is how to write a query that will traverse down one node of the tree.
And and this is especially useful if you're building something like Reddit comments that can be threaded multiple levels deep, or if you have a hierarchy of categories and only want to query a specific node in that tree.
We can do this with all of the documents in the same collection.
And you can see here at the top of the tree, we have the document a, and it has a parent value set to false.
That would be our top level comment.
And then let's say a user responds to this comment, that will be our b document, and it has a parent value set to a.
Now let's imagine we have a response to the b document, then we would have a c and d document, and they both reference the same parents of a, b.
So So what we've done here is create a composite key where the items at the 3rd level of the tree reference the IDs of the parent documents at the first and second levels.
These don't need to be an alphabetical order or anything like that.
You can just use Firestore's automatically generated IDs.
The only thing that's important is that the composite string is created in the same order that the parent elements appear in the tree.
Let's take a look at how we can actually traverse this tree.
If we just wanna grab the top level of the tree, which you would probably wanna do if you're showing the top level comments, is just query where the parent equals falls.
Now if you want to query across the breadth of the tree, or just get all of the top level responses or something like that, you can also save a level property on each document, which would allow you to do so.
But I think most use cases will require you to query the depth of the tree, so this part's technically optional.
Then assuming you have the level property, which is just an integer that increases by 1 for each to level, you can use where to query where all the documents live at a certain level, or you could use a range operator to get everything above or below a certain level.
Now things get a lot more interesting when we query the depth of the tree.
The composite key that we set up earlier will get larger and larger as we get deeper in the tree.
That means we can take a document ID and use it as a starting point and then query all of the children that start with that same ID in their composite parent string.
We can make this query by saying where the parent is greater than or equal to the ID and where the parent is less than or equal to the ID plus a high Unicode character.
So that would allow us to start from any node in the tree and then traverse downward.
And this is actually the same way that geohashing works, and it's a very powerful feature but is a little more advanced.
Now moving on to our next data model, which comes from Stephan.
We're going to look at how we can query a collection starting with an array of document IDs.
His data model was a lot more complex, but I just wanted to pick out one little thing that I think is helpful to a lot of people.
When working with Firestore, it's ideal to denormalize your data or embed it on single document, but there are a lot of cases when it's just not practical or possible to do that.
So one thing you can do is create a more normalized model like we see here with a sizes array, in each element in that array is a string referencing a document in a different collection.
But because there are no server side joins in Firestore, how do we actually join this data so we can use it in our UI?
We we can actually do this very efficiently in Firestore because we can send multiple read requests to the database, and Firestore will handle all of those requests concurrently.
Here's a little helper function you can use in JavaScript to join an array of IDs to a collection.
It takes a collection and an array of IDs as its argument, then it maps the IDs to a document read.
That will give us an array of promises, and then we can use promise dot all to resolve all of those promises concurrently.
And lastly, we can map the document snapshot to its raw data.
Now if we wanted to use this helper method in our code, we can just pass it this collection reference and the array of IDs that we wanna read, and it will return an array of the document data.
So if you want to model your data in a more normalized SQL style, you can do it using something like this.
Now that brings us to the final model coming from Troy, and he's building something like a social media style follower feed.
In other words, you follow a bunch of users and you wanna see the most recent posts from those users that you follow.
This is actually a very challenging requirement, so I built a full demo to show that it's possible in Firestore.
As you can see here, I'm logged in as Jeff d 23.
And when I unfollow these users, their posts are removed from my feed.
So, basically, what we're doing here is we're having the most recent posts from these users and then ordering them by date.
And on top of that, we're also maintaining the user to user follow relationship.
Let's start by taking a look at the data model.
The only way to make a system like this scale to a decent amount of users is to duplicate some data.
In our database, we have 3 collections at play.
We have one for users, one for posts that can be posted by a user and then another one for followers.
All the relational data will be handled by the followers collection.
And in fact, all of the data that you see in the UI is coming from a query from this followers document.
The followers document belongs to the user who is being followed and keeps track of all of the followers was in an array.
But in addition to keeping track of the followers, it also duplicates some of the user's recent posts.
It doesn't duplicate everything.
It only duplicates the data that we need is showing the feed, like the title and then maybe a preview of some of the text.
And the last thing we need to do is keep a time stamp of the user's last post.
What we're going to do is make a query on this last post property along with the user's array, and the result of that will be all of these duplicated posts from the users that the user is following.
Now an important thing to point out here is that we're keeping track of the followers on a single document, so that means we can only scale up to 1 megabyte of data.
And that means you may need to break this up into multiple documents as your app grows.
I'm going to go ahead and write this query inside of an async function because we'll need to do some data wrangling after we retrieve the initial data.
First, we'll make a reference to the followers collection and then we'll query it using array contains with the username of the logged in user.
After that, we'll make this a compound query by using order by with the last post time stamp.
And keep in mind this is a compound query, so it's going to require an index, and Firestore will give you a warning about that in the console.
Now the cool thing about this is we can limit it to 10 document reads, so it's a very efficient read, but it can provide us with data to populate potentially dozens of different posts in the feed.
Once we have the result of this query, we have an array that we need to then map to the document data.
And then on the document data, we're going to have an array of the user's most recent posts, and that's what we actually want to be showing in the view.
So I'm going to reduce this array of documents down to a new array that only contains the recent posts, and then I want to sort those recent posts based on their publish date.
We can use array sort to handle that.
And at this point, we now have a sorted array of posts that spans across multiple users that this user is following.
I'm I'm gonna go ahead and wrap things up there.
If you wanna learn more about data modeling and how these particular data models work, consider becoming a pro member at Fireship IO to get access to the course.
Thanks for watching, and I will talk to you soon.

Speaker 0: One of the most challenging aspects of building any app is determining the optimal data structure. Ideally, you want to maximize performance while minimizing costs. But that's easier said than done, and especially so with a NoSQL database like Firestore, because you really have to plan ahead on how you model your data. Last week, I asked everybody on our Slack channel to send me the biggest challenges that they faced modeling data in Firestore, store, and this conversation snowballed into an entire new course on Fireship dedicated completely to NoSQL data modeling and queries. So if this is a topic you're interested in, consider becoming a Pro Member to get access to that course, and I'll leave a discount code in the video description.

In today's video, we're going to look at 5 different ways you can structure and query your data that you may not know about. And each technique that we'll look at is based on a real submission from the community. A minute ago I said that you have to plan ahead when structuring your data. So what do I mean by that exactly? Well, the basic idea is that you want to pre render your data so that it fits the view or the screen in your app as closely as possible.

Ideally, you only need to make one document read or one query to a collection to fill the entire view with data. This differs from a relational SQL database where the idea is to break your data into a lot small pieces, and then join it all together server side. Now I've heard people say that Firestore or NoSQL databases in general are not good for relational data modeling, but that's actually not true, and it's really just a matter of rethinking the way that you model data in general. And what you'll find is that you can solve all the same problems that you can with SQL, but you can do it in a way is faster, less expensive, and more flexible. With that being said, let's go ahead and jump into our first data model and access pattern.

This suggestion comes from Somrat, and he wants to query a collection based on whether or not a user was mentioned in a post in that collection. This is similar to something like Twitter where if you mention a username the at symbol, it will notify that user that they've been mentioned, and they also might want to see a list of all the tweets that they've been mentioned in. In this case, we have a many to many relationship where a tweet can have many mentioned users, and a user can be mentioned in many tweets. But the thing to keep in mind here is that a tweet or a post can maybe only have 1 to 10 mentioned users, but a user can be mentioned in potentially billions of tweets. In situations like this, it's usually best to have the document with the smaller side of the relational data manage the relationships.

In in this example, we have an array of usernames embedded on the tweet or post document. You'd implement this logic somewhere in your front end code using regex is to extract the usernames out of the post text and then duplicate them on the same document as an array. Now it becomes very easy to query all the tweets that a user is mentioned in by making a reference to the post collection, then we can use the array contains operator to get all the tweets that have a corresponding user ID in that mentions array. Then we could use a cloud function to listen for a new post to be created and then send a push notification or an email to your user when they're mentioned. So the main limitation what array contains is that you can only query for 1 item at a time.

And that leads me to the next challenge, which is a category system that allows you to filter by multiple categories at the same time while also being able to see if a post contains a given category out of potentially multiple past values and also the ability to filter by not containing a certain value. So these requirements roughly translate into an and or or not statement in a SQL database. I'm gonna go ahead and call these tags instead of categories, but the general idea here is the same. We have a tags collection, which is really just there to hold any extra data about a tag, like a description or a URL or something like that. And I'd recommend giving your tags a unique ID that's something descriptive that you can actually show in the UI.

And then you'll take that ID and associate it to a post by using a map. So we have a map called tags, and then each key in that map is the ID from the tags collection. And the cool thing about Firestore is that it automatically indexes the keys in the map, which means we can query them without any additional configuration. First, let's take a look at how we can do a logical inquiry, or in other words, we want to get all of the items that have both of these tags present. We can do this by simply chaining together where statements.

And as long as they're using the equal to operator, we can do this as many times as we want. So this works great if you're filtering by multiple Boolean values. But if you throw in a range operator, that means you'll need a composite index. And keep in mind, you can only use 1 range operator per query, so that's a limitation that you'll want to consider. Now if you wanted to make something that was more like an or query, you can just make multiple queries for separate tags.

We're able to run these queries concurrently, and then we can just join them together and filter throughout any duplicates in our client side code. Now the most difficult part of this challenge is implementing the not logic. The values on the map are set up as booleans, so the most straightforward approach here would be to add all of the tags to every document and then set them to false by default. This would be perfectly fine if you have a finite set of tags and know their values up front, but it would be much more challenging if tags are generated by users and there could be potentially billions of tags. So that's a limitation to consider.

But keep in mind that if you have a numeric value or if you have a string value with some sort of ordering embedded in it, then you can simulate a knot query by using range operators. For example, if we wanted to get all of the items that were not $20, then we could query on either side of that range. Or in other words, all the items that are less than $20 and all the items that are more than $20. And if you're doing a lot of stuff like this, it might be time to look into something like Algolia to index some of your data in a full text search engine. That's what I do for Fireship, and Firebase and Algolia work really well together.

Now moving on to our next data model, this one comes from Lenny Cunningham and his flight now dot net app. Geolocation queries bring up a really interesting data modeling technique called composite strings. Not only do composite strings allow us to do things like geohashing, but it also allows us to do things like tree traversals and threaded comments. Take a look at this tree structure we have here where each letter represents a document in the database. What I'm going to show you next is how to write a query that will traverse down one node of the tree.

And and this is especially useful if you're building something like Reddit comments that can be threaded multiple levels deep, or if you have a hierarchy of categories and only want to query a specific node in that tree. We can do this with all of the documents in the same collection. And you can see here at the top of the tree, we have the document a, and it has a parent value set to false. That would be our top level comment. And then let's say a user responds to this comment, that will be our b document, and it has a parent value set to a.

Now let's imagine we have a response to the b document, then we would have a c and d document, and they both reference the same parents of a, b. So So what we've done here is create a composite key where the items at the 3rd level of the tree reference the IDs of the parent documents at the first and second levels. These don't need to be an alphabetical order or anything like that. You can just use Firestore's automatically generated IDs. The only thing that's important is that the composite string is created in the same order that the parent elements appear in the tree.

Let's take a look at how we can actually traverse this tree. If we just wanna grab the top level of the tree, which you would probably wanna do if you're showing the top level comments, is just query where the parent equals falls. Now if you want to query across the breadth of the tree, or just get all of the top level responses or something like that, you can also save a level property on each document, which would allow you to do so. But I think most use cases will require you to query the depth of the tree, so this part's technically optional. Then assuming you have the level property, which is just an integer that increases by 1 for each to level, you can use where to query where all the documents live at a certain level, or you could use a range operator to get everything above or below a certain level.

Now things get a lot more interesting when we query the depth of the tree. The composite key that we set up earlier will get larger and larger as we get deeper in the tree. That means we can take a document ID and use it as a starting point and then query all of the children that start with that same ID in their composite parent string. We can make this query by saying where the parent is greater than or equal to the ID and where the parent is less than or equal to the ID plus a high Unicode character. So that would allow us to start from any node in the tree and then traverse downward.

And this is actually the same way that geohashing works, and it's a very powerful feature but is a little more advanced. Now moving on to our next data model, which comes from Stephan. We're going to look at how we can query a collection starting with an array of document IDs. His data model was a lot more complex, but I just wanted to pick out one little thing that I think is helpful to a lot of people. When working with Firestore, it's ideal to denormalize your data or embed it on single document, but there are a lot of cases when it's just not practical or possible to do that.

So one thing you can do is create a more normalized model like we see here with a sizes array, in each element in that array is a string referencing a document in a different collection. But because there are no server side joins in Firestore, how do we actually join this data so we can use it in our UI? We we can actually do this very efficiently in Firestore because we can send multiple read requests to the database, and Firestore will handle all of those requests concurrently. Here's a little helper function you can use in JavaScript to join an array of IDs to a collection. It takes a collection and an array of IDs as its argument, then it maps the IDs to a document read.

That will give us an array of promises, and then we can use promise dot all to resolve all of those promises concurrently. And lastly, we can map the document snapshot to its raw data. Now if we wanted to use this helper method in our code, we can just pass it this collection reference and the array of IDs that we wanna read, and it will return an array of the document data. So if you want to model your data in a more normalized SQL style, you can do it using something like this. Now that brings us to the final model coming from Troy, and he's building something like a social media style follower feed.

In other words, you follow a bunch of users and you wanna see the most recent posts from those users that you follow. This is actually a very challenging requirement, so I built a full demo to show that it's possible in Firestore. As you can see here, I'm logged in as Jeff d 23. And when I unfollow these users, their posts are removed from my feed. So, basically, what we're doing here is we're having the most recent posts from these users and then ordering them by date.

And on top of that, we're also maintaining the user to user follow relationship. Let's start by taking a look at the data model. The only way to make a system like this scale to a decent amount of users is to duplicate some data. In our database, we have 3 collections at play. We have one for users, one for posts that can be posted by a user and then another one for followers.

All the relational data will be handled by the followers collection. And in fact, all of the data that you see in the UI is coming from a query from this followers document. The followers document belongs to the user who is being followed and keeps track of all of the followers was in an array. But in addition to keeping track of the followers, it also duplicates some of the user's recent posts. It doesn't duplicate everything.

It only duplicates the data that we need is showing the feed, like the title and then maybe a preview of some of the text. And the last thing we need to do is keep a time stamp of the user's last post. What we're going to do is make a query on this last post property along with the user's array, and the result of that will be all of these duplicated posts from the users that the user is following. Now an important thing to point out here is that we're keeping track of the followers on a single document, so that means we can only scale up to 1 megabyte of data. And that means you may need to break this up into multiple documents as your app grows.

I'm going to go ahead and write this query inside of an async function because we'll need to do some data wrangling after we retrieve the initial data. First, we'll make a reference to the followers collection and then we'll query it using array contains with the username of the logged in user. After that, we'll make this a compound query by using order by with the last post time stamp. And keep in mind this is a compound query, so it's going to require an index, and Firestore will give you a warning about that in the console. Now the cool thing about this is we can limit it to 10 document reads, so it's a very efficient read, but it can provide us with data to populate potentially dozens of different posts in the feed.

Once we have the result of this query, we have an array that we need to then map to the document data. And then on the document data, we're going to have an array of the user's most recent posts, and that's what we actually want to be showing in the view. So I'm going to reduce this array of documents down to a new array that only contains the recent posts, and then I want to sort those recent posts based on their publish date. We can use array sort to handle that. And at this point, we now have a sorted array of posts that spans across multiple users that this user is following.

I'm I'm gonna go ahead and wrap things up there. If you wanna learn more about data modeling and how these particular data models work, consider becoming a pro member at Fireship IO to get access to the course. Thanks for watching, and I will talk to you soon.
One of the most challenging aspects of building any app is determining the optimal data structure. Ideally, you want to maximize performance while minimizing costs. But that's easier said than done, and especially so with a NoSQL database like Firestore, because you really have to plan ahead on how you model your data. Last week, I asked everybody on our Slack channel to send me the biggest challenges that they faced modeling data in Firestore, store, and this conversation snowballed into an entire new course on Fireship dedicated completely to NoSQL data modeling and queries. So if this is a topic you're interested in, consider becoming a Pro Member to get access to that course, and I'll leave a discount code in the video description. In today's video, we're going to look at 5 different ways you can structure and query your data that you may not know about. And each technique that we'll look at is based on a real submission from the community. A minute ago I said that you have to plan ahead when structuring your data. So what do I mean by that exactly? Well, the basic idea is that you want to pre render your data so that it fits the view or the screen in your app as closely as possible. Ideally, you only need to make one document read or one query to a collection to fill the entire view with data. This differs from a relational SQL database where the idea is to break your data into a lot small pieces, and then join it all together server side. Now I've heard people say that Firestore or NoSQL databases in general are not good for relational data modeling, but that's actually not true, and it's really just a matter of rethinking the way that you model data in general. And what you'll find is that you can solve all the same problems that you can with SQL, but you can do it in a way is faster, less expensive, and more flexible. With that being said, let's go ahead and jump into our first data model and access pattern. This suggestion comes from Somrat, and he wants to query a collection based on whether or not a user was mentioned in a post in that collection. This is similar to something like Twitter where if you mention a username the at symbol, it will notify that user that they've been mentioned, and they also might want to see a list of all the tweets that they've been mentioned in. In this case, we have a many to many relationship where a tweet can have many mentioned users, and a user can be mentioned in many tweets. But the thing to keep in mind here is that a tweet or a post can maybe only have 1 to 10 mentioned users, but a user can be mentioned in potentially billions of tweets. In situations like this, it's usually best to have the document with the smaller side of the relational data manage the relationships.
databases
data analysis
twitter
data management
data
modeling
visualization
social media
machine learning
blogging
architecture
programming
customer support
In in this example, we have an array of usernames embedded on the tweet or post document. You'd implement this logic somewhere in your front end code using regex is to extract the usernames out of the post text and then duplicate them on the same document as an array.
Now it becomes very easy to query all the tweets that a user is mentioned in by making a reference to the post collection, then we can use the array contains operator to get all the tweets that have a corresponding user ID in that mentions array. Then we could use a cloud function to listen for a new post to be created and then send a push notification or an email to your user when they're mentioned.
twitter
So the main limitation what array contains is that you can only query for 1 item at a time. And that leads me to the next challenge, which is a category system that allows you to filter by multiple categories at the same time while also being able to see if a post contains a given category out of potentially multiple past values and also the ability to filter by not containing a certain value.
So these requirements roughly translate into an and or or not statement in a SQL database. I'm gonna go ahead and call these tags instead of categories, but the general idea here is the same. We have a tags collection, which is really just there to hold any extra data about a tag, like a description or a URL or something like that.
databases
data management
And I'd recommend giving your tags a unique ID that's something descriptive that you can actually show in the UI. And then you'll take that ID and associate it to a post by using a map.
So we have a map called tags, and then each key in that map is the ID from the tags collection. And the cool thing about Firestore is that it automatically indexes the keys in the map, which means we can query them without any additional configuration. First, let's take a look at how we can do a logical inquiry, or in other words, we want to get all of the items that have both of these tags present. We can do this by simply chaining together where statements. And as long as they're using the equal to operator, we can do this as many times as we want. So this works great if you're filtering by multiple Boolean values. But if you throw in a range operator, that means you'll need a composite index.
logic
logical reasoning
operations
google
And keep in mind, you can only use 1 range operator per query, so that's a limitation that you'll want to consider. Now if you wanted to make something that was more like an or query, you can just make multiple queries for separate tags.
We're able to run these queries concurrently, and then we can just join them together and filter throughout any duplicates in our client side code. Now the most difficult part of this challenge is implementing the not logic. The values on the map are set up as booleans, so the most straightforward approach here would be to add all of the tags to every document and then set them to false by default.
programming
databases
This would be perfectly fine if you have a finite set of tags and know their values up front, but it would be much more challenging if tags are generated by users and there could be potentially billions of tags. So that's a limitation to consider. But keep in mind that if you have a numeric value or if you have a string value with some sort of ordering embedded in it, then you can simulate a knot query by using range operators. For example, if we wanted to get all of the items that were not $20, then we could query on either side of that range. Or in other words, all the items that are less than $20 and all the items that are more than $20. And if you're doing a lot of stuff like this, it might be time to look into something like Algolia to index some of your data in a full text search engine. That's what I do for Fireship, and Firebase and Algolia work really well together. Now moving on to our next data model, this one comes from Lenny Cunningham and his flight now dot net app.
Geolocation queries bring up a really interesting data modeling technique called composite strings. Not only do composite strings allow us to do things like geohashing, but it also allows us to do things like tree traversals and threaded comments. Take a look at this tree structure we have here where each letter represents a document in the database. What I'm going to show you next is how to write a query that will traverse down one node of the tree. And and this is especially useful if you're building something like Reddit comments that can be threaded multiple levels deep, or if you have a hierarchy of categories and only want to query a specific node in that tree. We can do this with all of the documents in the same collection. And you can see here at the top of the tree, we have the document a, and it has a parent value set to false. That would be our top level comment. And then let's say a user responds to this comment, that will be our b document, and it has a parent value set to a.
databases
data
trees
programming
documentation
Now let's imagine we have a response to the b document, then we would have a c and d document, and they both reference the same parents of a, b. So So what we've done here is create a composite key where the items at the 3rd level of the tree reference the IDs of the parent documents at the first and second levels.
documentation
These don't need to be an alphabetical order or anything like that. You can just use Firestore's automatically generated IDs. The only thing that's important is that the composite string is created in the same order that the parent elements appear in the tree.
Let's take a look at how we can actually traverse this tree. If we just wanna grab the top level of the tree, which you would probably wanna do if you're showing the top level comments, is just query where the parent equals falls. Now if you want to query across the breadth of the tree, or just get all of the top level responses or something like that, you can also save a level property on each document, which would allow you to do so. But I think most use cases will require you to query the depth of the tree, so this part's technically optional.
trees
forests
databases
Then assuming you have the level property, which is just an integer that increases by 1 for each to level, you can use where to query where all the documents live at a certain level, or you could use a range operator to get everything above or below a certain level. Now things get a lot more interesting when we query the depth of the tree. The composite key that we set up earlier will get larger and larger as we get deeper in the tree.
That means we can take a document ID and use it as a starting point and then query all of the children that start with that same ID in their composite parent string. We can make this query by saying where the parent is greater than or equal to the ID and where the parent is less than or equal to the ID plus a high Unicode character.
children
So that would allow us to start from any node in the tree and then traverse downward. And this is actually the same way that geohashing works, and it's a very powerful feature but is a little more advanced. Now moving on to our next data model, which comes from Stephan. We're going to look at how we can query a collection starting with an array of document IDs. His data model was a lot more complex, but I just wanted to pick out one little thing that I think is helpful to a lot of people. When working with Firestore, it's ideal to denormalize your data or embed it on single document, but there are a lot of cases when it's just not practical or possible to do that.
data
data analysis
data management
databases
geology
trees
geometry
So one thing you can do is create a more normalized model like we see here with a sizes array, in each element in that array is a string referencing a document in a different collection. But because there are no server side joins in Firestore, how do we actually join this data so we can use it in our UI?
databases
We we can actually do this very efficiently in Firestore because we can send multiple read requests to the database, and Firestore will handle all of those requests concurrently. Here's a little helper function you can use in JavaScript to join an array of IDs to a collection. It takes a collection and an array of IDs as its argument, then it maps the IDs to a document read.
databases
That will give us an array of promises, and then we can use promise dot all to resolve all of those promises concurrently. And lastly, we can map the document snapshot to its raw data.
Now if we wanted to use this helper method in our code, we can just pass it this collection reference and the array of IDs that we wanna read, and it will return an array of the document data. So if you want to model your data in a more normalized SQL style, you can do it using something like this. Now that brings us to the final model coming from Troy, and he's building something like a social media style follower feed. In other words, you follow a bunch of users and you wanna see the most recent posts from those users that you follow. This is actually a very challenging requirement, so I built a full demo to show that it's possible in Firestore. As you can see here, I'm logged in as Jeff d 23.
databases
data management
data
modeling
blogging
programming
And when I unfollow these users, their posts are removed from my feed.
So, basically, what we're doing here is we're having the most recent posts from these users and then ordering them by date. And on top of that, we're also maintaining the user to user follow relationship. Let's start by taking a look at the data model. The only way to make a system like this scale to a decent amount of users is to duplicate some data. In our database, we have 3 collections at play. We have one for users, one for posts that can be posted by a user and then another one for followers. All the relational data will be handled by the followers collection. And in fact, all of the data that you see in the UI is coming from a query from this followers document.
databases
data management
data analysis
data
modeling
The followers document belongs to the user who is being followed and keeps track of all of the followers was in an array. But in addition to keeping track of the followers, it also duplicates some of the user's recent posts. It doesn't duplicate everything. It only duplicates the data that we need is showing the feed, like the title and then maybe a preview of some of the text. And the last thing we need to do is keep a time stamp of the user's last post. What we're going to do is make a query on this last post property along with the user's array, and the result of that will be all of these duplicated posts from the users that the user is following. Now an important thing to point out here is that we're keeping track of the followers on a single document, so that means we can only scale up to 1 megabyte of data. And that means you may need to break this up into multiple documents as your app grows. I'm going to go ahead and write this query inside of an async function because we'll need to do some data wrangling after we retrieve the initial data. First, we'll make a reference to the followers collection and then we'll query it using array contains with the username of the logged in user. After that, we'll make this a compound query by using order by with the last post time stamp. And keep in mind this is a compound query, so it's going to require an index, and Firestore will give you a warning about that in the console.
Now the cool thing about this is we can limit it to 10 document reads, so it's a very efficient read, but it can provide us with data to populate potentially dozens of different posts in the feed. Once we have the result of this query, we have an array that we need to then map to the document data.
databases
reading
And then on the document data, we're going to have an array of the user's most recent posts, and that's what we actually want to be showing in the view. So I'm going to reduce this array of documents down to a new array that only contains the recent posts, and then I want to sort those recent posts based on their publish date.
We can use array sort to handle that. And at this point, we now have a sorted array of posts that spans across multiple users that this user is following. I'm I'm gonna go ahead and wrap things up there. If you wanna learn more about data modeling and how these particular data models work, consider becoming a pro member at Fireship IO to get access to the course. Thanks for watching, and I will talk to you soon.
data management
databases
data
data analysis
One of the most challenging aspects of building any app is determining the optimal data structure. Ideally, you want to maximize performance while minimizing costs. But that's easier said than done, and especially so with a NoSQL database like Firestore, because you really have to plan ahead on how you model your data. Last week, I asked everybody on our Slack channel to send me the biggest challenges that they faced modeling data in Firestore, store, and this conversation snowballed into an entire new course on Fireship dedicated completely to NoSQL data modeling and queries. So if this is a topic you're interested in, consider becoming a Pro Member to get access to that course, and I'll leave a discount code in the video description. In today's video, we're going to look at 5 different ways you can structure and query your data that you may not know about. And each technique that we'll look at is based on a real submission from the community. A minute ago I said that you have to plan ahead when structuring your data. So what do I mean by that exactly? Well, the basic idea is that you want to pre render your data so that it fits the view or the screen in your app as closely as possible. Ideally, you only need to make one document read or one query to a collection to fill the entire view with data. This differs from a relational SQL database where the idea is to break your data into a lot small pieces, and then join it all together server side. Now I've heard people say that Firestore or NoSQL databases in general are not good for relational data modeling, but that's actually not true, and it's really just a matter of rethinking the way that you model data in general. And what you'll find is that you can solve all the same problems that you can with SQL, but you can do it in a way is faster, less expensive, and more flexible. With that being said, let's go ahead and jump into our first data model and access pattern. This suggestion comes from Somrat, and he wants to query a collection based on whether or not a user was mentioned in a post in that collection. This is similar to something like Twitter where if you mention a username the at symbol, it will notify that user that they've been mentioned, and they also might want to see a list of all the tweets that they've been mentioned in. In this case, we have a many to many relationship where a tweet can have many mentioned users, and a user can be mentioned in many tweets. But the thing to keep in mind here is that a tweet or a post can maybe only have 1 to 10 mentioned users, but a user can be mentioned in potentially billions of tweets. In situations like this, it's usually best to have the document with the smaller side of the relational data manage the relationships. In in this example, we have an array of usernames embedded on the tweet or post document. You'd implement this logic somewhere in your front end code using regex is to extract the usernames out of the post text and then duplicate them on the same document as an array. Now it becomes very easy to query all the tweets that a user is mentioned in by making a reference to the post collection, then we can use the array contains operator to get all the tweets that have a corresponding user ID in that mentions array. Then we could use a cloud function to listen for a new post to be created and then send a push notification or an email to your user when they're mentioned. So the main limitation what array contains is that you can only query for 1 item at a time. And that leads me to the next challenge, which is a category system that allows you to filter by multiple categories at the same time while also being able to see if a post contains a given category out of potentially multiple past values and also the ability to filter by not containing a certain value. So these requirements roughly translate into an and or or not statement in a SQL database. I'm gonna go ahead and call these tags instead of categories, but the general idea here is the same. We have a tags collection, which is really just there to hold any extra data about a tag, like a description or a URL or something like that. And I'd recommend giving your tags a unique ID that's something descriptive that you can actually show in the UI. And then you'll take that ID and associate it to a post by using a map. So we have a map called tags, and then each key in that map is the ID from the tags collection. And the cool thing about Firestore is that it automatically indexes the keys in the map, which means we can query them without any additional configuration. First, let's take a look at how we can do a logical inquiry, or in other words, we want to get all of the items that have both of these tags present. We can do this by simply chaining together where statements. And as long as they're using the equal to operator, we can do this as many times as we want. So this works great if you're filtering by multiple Boolean values. But if you throw in a range operator, that means you'll need a composite index. And keep in mind, you can only use 1 range operator per query, so that's a limitation that you'll want to consider. Now if you wanted to make something that was more like an or query, you can just make multiple queries for separate tags. We're able to run these queries concurrently, and then we can just join them together and filter throughout any duplicates in our client side code. Now the most difficult part of this challenge is implementing the not logic. The values on the map are set up as booleans, so the most straightforward approach here would be to add all of the tags to every document and then set them to false by default. This would be perfectly fine if you have a finite set of tags and know their values up front, but it would be much more challenging if tags are generated by users and there could be potentially billions of tags. So that's a limitation to consider. But keep in mind that if you have a numeric value or if you have a string value with some sort of ordering embedded in it, then you can simulate a knot query by using range operators. For example, if we wanted to get all of the items that were not $20, then we could query on either side of that range. Or in other words, all the items that are less than $20 and all the items that are more than $20. And if you're doing a lot of stuff like this, it might be time to look into something like Algolia to index some of your data in a full text search engine. That's what I do for Fireship, and Firebase and Algolia work really well together. Now moving on to our next data model, this one comes from Lenny Cunningham and his flight now dot net app. Geolocation queries bring up a really interesting data modeling technique called composite strings. Not only do composite strings allow us to do things like geohashing, but it also allows us to do things like tree traversals and threaded comments. Take a look at this tree structure we have here where each letter represents a document in the database. What I'm going to show you next is how to write a query that will traverse down one node of the tree. And and this is especially useful if you're building something like Reddit comments that can be threaded multiple levels deep, or if you have a hierarchy of categories and only want to query a specific node in that tree. We can do this with all of the documents in the same collection. And you can see here at the top of the tree, we have the document a, and it has a parent value set to false. That would be our top level comment. And then let's say a user responds to this comment, that will be our b document, and it has a parent value set to a. Now let's imagine we have a response to the b document, then we would have a c and d document, and they both reference the same parents of a, b. So So what we've done here is create a composite key where the items at the 3rd level of the tree reference the IDs of the parent documents at the first and second levels. These don't need to be an alphabetical order or anything like that. You can just use Firestore's automatically generated IDs. The only thing that's important is that the composite string is created in the same order that the parent elements appear in the tree. Let's take a look at how we can actually traverse this tree. If we just wanna grab the top level of the tree, which you would probably wanna do if you're showing the top level comments, is just query where the parent equals falls. Now if you want to query across the breadth of the tree, or just get all of the top level responses or something like that, you can also save a level property on each document, which would allow you to do so. But I think most use cases will require you to query the depth of the tree, so this part's technically optional. Then assuming you have the level property, which is just an integer that increases by 1 for each to level, you can use where to query where all the documents live at a certain level, or you could use a range operator to get everything above or below a certain level. Now things get a lot more interesting when we query the depth of the tree. The composite key that we set up earlier will get larger and larger as we get deeper in the tree. That means we can take a document ID and use it as a starting point and then query all of the children that start with that same ID in their composite parent string. We can make this query by saying where the parent is greater than or equal to the ID and where the parent is less than or equal to the ID plus a high Unicode character. So that would allow us to start from any node in the tree and then traverse downward. And this is actually the same way that geohashing works, and it's a very powerful feature but is a little more advanced. Now moving on to our next data model, which comes from Stephan. We're going to look at how we can query a collection starting with an array of document IDs. His data model was a lot more complex, but I just wanted to pick out one little thing that I think is helpful to a lot of people. When working with Firestore, it's ideal to denormalize your data or embed it on single document, but there are a lot of cases when it's just not practical or possible to do that. So one thing you can do is create a more normalized model like we see here with a sizes array, in each element in that array is a string referencing a document in a different collection. But because there are no server side joins in Firestore, how do we actually join this data so we can use it in our UI? We we can actually do this very efficiently in Firestore because we can send multiple read requests to the database, and Firestore will handle all of those requests concurrently. Here's a little helper function you can use in JavaScript to join an array of IDs to a collection. It takes a collection and an array of IDs as its argument, then it maps the IDs to a document read. That will give us an array of promises, and then we can use promise dot all to resolve all of those promises concurrently. And lastly, we can map the document snapshot to its raw data. Now if we wanted to use this helper method in our code, we can just pass it this collection reference and the array of IDs that we wanna read, and it will return an array of the document data. So if you want to model your data in a more normalized SQL style, you can do it using something like this. Now that brings us to the final model coming from Troy, and he's building something like a social media style follower feed. In other words, you follow a bunch of users and you wanna see the most recent posts from those users that you follow. This is actually a very challenging requirement, so I built a full demo to show that it's possible in Firestore. As you can see here, I'm logged in as Jeff d 23. And when I unfollow these users, their posts are removed from my feed. So, basically, what we're doing here is we're having the most recent posts from these users and then ordering them by date. And on top of that, we're also maintaining the user to user follow relationship. Let's start by taking a look at the data model. The only way to make a system like this scale to a decent amount of users is to duplicate some data. In our database, we have 3 collections at play. We have one for users, one for posts that can be posted by a user and then another one for followers. All the relational data will be handled by the followers collection. And in fact, all of the data that you see in the UI is coming from a query from this followers document. The followers document belongs to the user who is being followed and keeps track of all of the followers was in an array. But in addition to keeping track of the followers, it also duplicates some of the user's recent posts. It doesn't duplicate everything. It only duplicates the data that we need is showing the feed, like the title and then maybe a preview of some of the text. And the last thing we need to do is keep a time stamp of the user's last post. What we're going to do is make a query on this last post property along with the user's array, and the result of that will be all of these duplicated posts from the users that the user is following. Now an important thing to point out here is that we're keeping track of the followers on a single document, so that means we can only scale up to 1 megabyte of data. And that means you may need to break this up into multiple documents as your app grows. I'm going to go ahead and write this query inside of an async function because we'll need to do some data wrangling after we retrieve the initial data. First, we'll make a reference to the followers collection and then we'll query it using array contains with the username of the logged in user. After that, we'll make this a compound query by using order by with the last post time stamp. And keep in mind this is a compound query, so it's going to require an index, and Firestore will give you a warning about that in the console. Now the cool thing about this is we can limit it to 10 document reads, so it's a very efficient read, but it can provide us with data to populate potentially dozens of different posts in the feed. Once we have the result of this query, we have an array that we need to then map to the document data. And then on the document data, we're going to have an array of the user's most recent posts, and that's what we actually want to be showing in the view. So I'm going to reduce this array of documents down to a new array that only contains the recent posts, and then I want to sort those recent posts based on their publish date. We can use array sort to handle that. And at this point, we now have a sorted array of posts that spans across multiple users that this user is following. I'm I'm gonna go ahead and wrap things up there. If you wanna learn more about data modeling and how these particular data models work, consider becoming a pro member at Fireship IO to get access to the course. Thanks for watching, and I will talk to you soon.
One
one
of
of
the
the
most
most
challenging
challenging
aspects
aspects
of
of
building
building
any
any
app
app
is
is
determining
determining
the
the
optimal
optimal
data
data
structure.
structure
Ideally,
ideally
you
you
want
want
to
to
maximize
maximize
performance
performance
while
while
minimizing
minimizing
costs.
costs
But
but
that's
that's
easier
easier
said
said
than
than
done,
done
and
and
especially
especially
so
so
with
with
a
a
NoSQL
nosql
database
database
like
like
Firestore,
firestore
because
because
you
you
really
really
have
have
to
to
plan
plan
ahead
ahead
on
on
how
how
you
you
model
model
your
your
data.
data
Last
last
week,
week
I
i
asked
asked
everybody
everybody
on
on
our
our
Slack
slack
channel
channel
to
to
send
send
me
me
the
the
biggest
biggest
challenges
challenges
that
that
they
they
faced
faced
modeling
modeling
data
data
in
in
Firestore,
firestore
store,
store
and
and
this
this
conversation
conversation
snowballed
snowballed
into
into
an
an
entire
entire
new
new
course
course
on
on
Fireship
fireship
dedicated
dedicated
completely
completely
to
to
NoSQL
nosql
data
data
modeling
modeling
and
and
queries.
queries
So
so
if
if
this
this
is
is
a
a
topic
topic
you're
you're
interested
interested
in,
in
consider
consider
becoming
becoming
a
a
Pro
pro
Member
member
to
to
get
get
access
access
to
to
that
that
course,
course
and
and
I'll
i'll
leave
leave
a
a
discount
discount
code
code
in
in
the
the
video
video
description.
description
In
in
today's
today's
video,
video
we're
we're
going
going
to
to
look
look
at
at
5
5
different
different
ways
ways
you
you
can
can
structure
structure
and
and
query
query
your
your
data
data
that
that
you
you
may
may
not
not
know
know
about.
about
And
and
each
each
technique
technique
that
that
we'll
we'll
look
look
at
at
is
is
based
based
on
on
a
a
real
real
submission
submission
from
from
the
the
community.
community
A
a
minute
minute
ago
ago
I
i
said
said
that
that
you
you
have
have
to
to
plan
plan
ahead
ahead
when
when
structuring
structuring
your
your
data.
data
So
so
what
what
do
do
I
i
mean
mean
by
by
that
that
exactly?
exactly
Well,
well
the
the
basic
basic
idea
idea
is
is
that
that
you
you
want
want
to
to
pre
pre
render
render
your
your
data
data
so
so
that
that
it
it
fits
fits
the
the
view
view
or
or
the
the
screen
screen
in
in
your
your
app
app
as
as
closely
closely
as
as
possible.
possible
Ideally,
ideally
you
you
only
only
need
need
to
to
make
make
one
one
document
document
read
read
or
or
one
one
query
query
to
to
a
a
collection
collection
to
to
fill
fill
the
the
entire
entire
view
view
with
with
data.
data
This
this
differs
differs
from
from
a
a
relational
relational
SQL
sql
database
database
where
where
the
the
idea
idea
is
is
to
to
break
break
your
your
data
data
into
into
a
a
lot
lot
small
small
pieces,
pieces
and
and
then
then
join
join
it
it
all
all
together
together
server
server
side.
side
Now
now
I've
i've
heard
heard
people
people
say
say
that
that
Firestore
firestore
or
or
NoSQL
nosql
databases
databases
in
in
general
general
are
are
not
not
good
good
for
for
relational
relational
data
data
modeling,
modeling
but
but
that's
that's
actually
actually
not
not
true,
true
and
and
it's
it's
really
really
just
just
a
a
matter
matter
of
of
rethinking
rethinking
the
the
way
way
that
that
you
you
model
model
data
data
in
in
general.
general
And
and
what
what
you'll
you'll
find
find
is
is
that
that
you
you
can
can
solve
solve
all
all
the
the
same
same
problems
problems
that
that
you
you
can
can
with
with
SQL,
sql
but
but
you
you
can
can
do
do
it
it
in
in
a
a
way
way
is
is
faster,
faster
less
less
expensive,
expensive
and
and
more
more
flexible.
flexible
With
with
that
that
being
being
said,
said
let's
let's
go
go
ahead
ahead
and
and
jump
jump
into
into
our
our
first
first
data
data
model
model
and
and
access
access
pattern.
pattern
This
this
suggestion
suggestion
comes
comes
from
from
Somrat,
somrat
and
and
he
he
wants
wants
to
to
query
query
a
a
collection
collection
based
based
on
on
whether
whether
or
or
not
not
a
a
user
user
was
was
mentioned
mentioned
in
in
a
a
post
post
in
in
that
that
collection.
collection
This
this
is
is
similar
similar
to
to
something
something
like
like
Twitter
twitter
where
where
if
if
you
you
mention
mention
a
a
username
username
the
the
at
at
symbol,
symbol
it
it
will
will
notify
notify
that
that
user
user
that
that
they've
they've
been
been
mentioned,
mentioned
and
and
they
they
also
also
might
might
want
want
to
to
see
see
a
a
list
list
of
of
all
all
the
the
tweets
tweets
that
that
they've
they've
been
been
mentioned
mentioned
in.
in
In
in
this
this
case,
case
we
we
have
have
a
a
many
many
to
to
many
many
relationship
relationship
where
where
a
a
tweet
tweet
can
can
have
have
many
many
mentioned
mentioned
users,
users
and
and
a
a
user
user
can
can
be
be
mentioned
mentioned
in
in
many
many
tweets.
tweets
But
but
the
the
thing
thing
to
to
keep
keep
in
in
mind
mind
here
here
is
is
that
that
a
a
tweet
tweet
or
or
a
a
post
post
can
can
maybe
maybe
only
only
have
have
1
1
to
to
10
10
mentioned
mentioned
users,
users
but
but
a
a
user
user
can
can
be
be
mentioned
mentioned
in
in
potentially
potentially
billions
billions
of
of
tweets.
tweets
In
in
situations
situations
like
like
this,
this
it's
it's
usually
usually
best
best
to
to
have
have
the
the
document
document
with
with
the
the
smaller
smaller
side
side
of
of
the
the
relational
relational
data
data
manage
manage
the
the
relationships.
relationships
In
in
in
in
this
this
example,
example
we
we
have
have
an
an
array
array
of
of
usernames
usernames
embedded
embedded
on
on
the
the
tweet
tweet
or
or
post
post
document.
document
You'd
you'd
implement
implement
this
this
logic
logic
somewhere
somewhere
in
in
your
your
front
front
end
end
code
code
using
using
regex
regex
is
is
to
to
extract
extract
the
the
usernames
usernames
out
out
of
of
the
the
post
post
text
text
and
and
then
then
duplicate
duplicate
them
them
on
on
the
the
same
same
document
document
as
as
an
an
array.
array
Now
now
it
it
becomes
becomes
very
very
easy
easy
to
to
query
query
all
all
the
the
tweets
tweets
that
that
a
a
user
user
is
is
mentioned
mentioned
in
in
by
by
making
making
a
a
reference
reference
to
to
the
the
post
post
collection,
collection
then
then
we
we
can
can
use
use
the
the
array
array
contains
contains
operator
operator
to
to
get
get
all
all
the
the
tweets
tweets
that
that
have
have
a
a
corresponding
corresponding
user
user
ID
id
in
in
that
that
mentions
mentions
array.
array
Then
then
we
we
could
could
use
use
a
a
cloud
cloud
function
function
to
to
listen
listen
for
for
a
a
new
new
post
post
to
to
be
be
created
created
and
and
then
then
send
send
a
a
push
push
notification
notification
or
or
an
an
email
email
to
to
your
your
user
user
when
when
they're
they're
mentioned.
mentioned
So
so
the
the
main
main
limitation
limitation
what
what
array
array
contains
contains
is
is
that
that
you
you
can
can
only
only
query
query
for
for
1
1
item
item
at
at
a
a
time.
time
And
and
that
that
leads
leads
me
me
to
to
the
the
next
next
challenge,
challenge
which
which
is
is
a
a
category
category
system
system
that
that
allows
allows
you
you
to
to
filter
filter
by
by
multiple
multiple
categories
categories
at
at
the
the
same
same
time
time
while
while
also
also
being
being
able
able
to
to
see
see
if
if
a
a
post
post
contains
contains
a
a
given
given
category
category
out
out
of
of
potentially
potentially
multiple
multiple
past
past
values
values
and
and
also
also
the
the
ability
ability
to
to
filter
filter
by
by
not
not
containing
containing
a
a
certain
certain
value.
value
So
so
these
these
requirements
requirements
roughly
roughly
translate
translate
into
into
an
an
and
and
or
or
or
or
not
not
statement
statement
in
in
a
a
SQL
sql
database.
database
I'm
i'm
gonna
gonna
go
go
ahead
ahead
and
and
call
call
these
these
tags
tags
instead
instead
of
of
categories,
categories
but
but
the
the
general
general
idea
idea
here
here
is
is
the
the
same.
same
We
we
have
have
a
a
tags
tags
collection,
collection
which
which
is
is
really
really
just
just
there
there
to
to
hold
hold
any
any
extra
extra
data
data
about
about
a
a
tag,
tag
like
like
a
a
description
description
or
or
a
a
URL
url
or
or
something
something
like
like
that.
that
And
and
I'd
i'd
recommend
recommend
giving
giving
your
your
tags
tags
a
a
unique
unique
ID
id
that's
that's
something
something
descriptive
descriptive
that
that
you
you
can
can
actually
actually
show
show
in
in
the
the
UI.
ui
And
and
then
then
you'll
you'll
take
take
that
that
ID
id
and
and
associate
associate
it
it
to
to
a
a
post
post
by
by
using
using
a
a
map.
map
So
so
we
we
have
have
a
a
map
map
called
called
tags,
tags
and
and
then
then
each
each
key
key
in
in
that
that
map
map
is
is
the
the
ID
id
from
from
the
the
tags
tags
collection.
collection
And
and
the
the
cool
cool
thing
thing
about
about
Firestore
firestore
is
is
that
that
it
it
automatically
automatically
indexes
indexes
the
the
keys
keys
in
in
the
the
map,
map
which
which
means
means
we
we
can
can
query
query
them
them
without
without
any
any
additional
additional
configuration.
configuration
First,
first
let's
let's
take
take
a
a
look
look
at
at
how
how
we
we
can
can
do
do
a
a
logical
logical
inquiry,
inquiry
or
or
in
in
other
other
words,
words
we
we
want
want
to
to
get
get
all
all
of
of
the
the
items
items
that
that
have
have
both
both
of
of
these
these
tags
tags
present.
present
We
we
can
can
do
do
this
this
by
by
simply
simply
chaining
chaining
together
together
where
where
statements.
statements
And
and
as
as
long
long
as
as
they're
they're
using
using
the
the
equal
equal
to
to
operator,
operator
we
we
can
can
do
do
this
this
as
as
many
many
times
times
as
as
we
we
want.
want
So
so
this
this
works
works
great
great
if
if
you're
you're
filtering
filtering
by
by
multiple
multiple
Boolean
boolean
values.
values
But
but
if
if
you
you
throw
throw
in
in
a
a
range
range
operator,
operator
that
that
means
means
you'll
you'll
need
need
a
a
composite
composite
index.
index
And
and
keep
keep
in
in
mind,
mind
you
you
can
can
only
only
use
use
1
1
range
range
operator
operator
per
per
query,
query
so
so
that's
that's
a
a
limitation
limitation
that
that
you'll
you'll
want
want
to
to
consider.
consider
Now
now
if
if
you
you
wanted
wanted
to
to
make
make
something
something
that
that
was
was
more
more
like
like
an
an
or
or
query,
query
you
you
can
can
just
just
make
make
multiple
multiple
queries
queries
for
for
separate
separate
tags.
tags
We're
we're
able
able
to
to
run
run
these
these
queries
queries
concurrently,
concurrently
and
and
then
then
we
we
can
can
just
just
join
join
them
them
together
together
and
and
filter
filter
throughout
throughout
any
any
duplicates
duplicates
in
in
our
our
client
client
side
side
code.
code
Now
now
the
the
most
most
difficult
difficult
part
part
of
of
this
this
challenge
challenge
is
is
implementing
implementing
the
the
not
not
logic.
logic
The
the
values
values
on
on
the
the
map
map
are
are
set
set
up
up
as
as
booleans,
booleans
so
so
the
the
most
most
straightforward
straightforward
approach
approach
here
here
would
would
be
be
to
to
add
add
all
all
of
of
the
the
tags
tags
to
to
every
every
document
document
and
and
then
then
set
set
them
them
to
to
false
false
by
by
default.
default
This
this
would
would
be
be
perfectly
perfectly
fine
fine
if
if
you
you
have
have
a
a
finite
finite
set
set
of
of
tags
tags
and
and
know
know
their
their
values
values
up
up
front,
front
but
but
it
it
would
would
be
be
much
much
more
more
challenging
challenging
if
if
tags
tags
are
are
generated
generated
by
by
users
users
and
and
there
there
could
could
be
be
potentially
potentially
billions
billions
of
of
tags.
tags
So
so
that's
that's
a
a
limitation
limitation
to
to
consider.
consider
But
but
keep
keep
in
in
mind
mind
that
that
if
if
you
you
have
have
a
a
numeric
numeric
value
value
or
or
if
if
you
you
have
have
a
a
string
string
value
value
with
with
some
some
sort
sort
of
of
ordering
ordering
embedded
embedded
in
in
it,
it
then
then
you
you
can
can
simulate
simulate
a
a
knot
knot
query
query
by
by
using
using
range
range
operators.
operators
For
for
example,
example
if
if
we
we
wanted
wanted
to
to
get
get
all
all
of
of
the
the
items
items
that
that
were
were
not
not
$20,
$20
then
then
we
we
could
could
query
query
on
on
either
either
side
side
of
of
that
that
range.
range
Or
or
in
in
other
other
words,
words
all
all
the
the
items
items
that
that
are
are
less
less
than
than
$20
$20
and
and
all
all
the
the
items
items
that
that
are
are
more
more
than
than
$20.
$20
And
and
if
if
you're
you're
doing
doing
a
a
lot
lot
of
of
stuff
stuff
like
like
this,
this
it
it
might
might
be
be
time
time
to
to
look
look
into
into
something
something
like
like
Algolia
algolia
to
to
index
index
some
some
of
of
your
your
data
data
in
in
a
a
full
full
text
text
search
search
engine.
engine
That's
that's
what
what
I
i
do
do
for
for
Fireship,
fireship
and
and
Firebase
firebase
and
and
Algolia
algolia
work
work
really
really
well
well
together.
together
Now
now
moving
moving
on
on
to
to
our
our
next
next
data
data
model,
model
this
this
one
one
comes
comes
from
from
Lenny
lenny
Cunningham
cunningham
and
and
his
his
flight
flight
now
now
dot
dot
net
net
app.
app
Geolocation
geolocation
queries
queries
bring
bring
up
up
a
a
really
really
interesting
interesting
data
data
modeling
modeling
technique
technique
called
called
composite
composite
strings.
strings
Not
not
only
only
do
do
composite
composite
strings
strings
allow
allow
us
us
to
to
do
do
things
things
like
like
geohashing,
geohashing
but
but
it
it
also
also
allows
allows
us
us
to
to
do
do
things
things
like
like
tree
tree
traversals
traversals
and
and
threaded
threaded
comments.
comments
Take
take
a
a
look
look
at
at
this
this
tree
tree
structure
structure
we
we
have
have
here
here
where
where
each
each
letter
letter
represents
represents
a
a
document
document
in
in
the
the
database.
database
What
what
I'm
i'm
going
going
to
to
show
show
you
you
next
next
is
is
how
how
to
to
write
write
a
a
query
query
that
that
will
will
traverse
traverse
down
down
one
one
node
node
of
of
the
the
tree.
tree
And
and
and
and
this
this
is
is
especially
especially
useful
useful
if
if
you're
you're
building
building
something
something
like
like
Reddit
reddit
comments
comments
that
that
can
can
be
be
threaded
threaded
multiple
multiple
levels
levels
deep,
deep
or
or
if
if
you
you
have
have
a
a
hierarchy
hierarchy
of
of
categories
categories
and
and
only
only
want
want
to
to
query
query
a
a
specific
specific
node
node
in
in
that
that
tree.
tree
We
we
can
can
do
do
this
this
with
with
all
all
of
of
the
the
documents
documents
in
in
the
the
same
same
collection.
collection
And
and
you
you
can
can
see
see
here
here
at
at
the
the
top
top
of
of
the
the
tree,
tree
we
we
have
have
the
the
document
document
a,
a
and
and
it
it
has
has
a
a
parent
parent
value
value
set
set
to
to
false.
false
That
that
would
would
be
be
our
our
top
top
level
level
comment.
comment
And
and
then
then
let's
let's
say
say
a
a
user
user
responds
responds
to
to
this
this
comment,
comment
that
that
will
will
be
be
our
our
b
b
document,
document
and
and
it
it
has
has
a
a
parent
parent
value
value
set
set
to
to
a.
a
Now
now
let's
let's
imagine
imagine
we
we
have
have
a
a
response
response
to
to
the
the
b
b
document,
document
then
then
we
we
would
would
have
have
a
a
c
c
and
and
d
d
document,
document
and
and
they
they
both
both
reference
reference
the
the
same
same
parents
parents
of
of
a,
a
b.
b
So
so
So
so
what
what
we've
we've
done
done
here
here
is
is
create
create
a
a
composite
composite
key
key
where
where
the
the
items
items
at
at
the
the
3rd
3rd
level
level
of
of
the
the
tree
tree
reference
reference
the
the
IDs
ids
of
of
the
the
parent
parent
documents
documents
at
at
the
the
first
first
and
and
second
second
levels.
levels
These
these
don't
don't
need
need
to
to
be
be
an
an
alphabetical
alphabetical
order
order
or
or
anything
anything
like
like
that.
that
You
you
can
can
just
just
use
use
Firestore's
firestore's
automatically
automatically
generated
generated
IDs.
ids
The
the
only
only
thing
thing
that's
that's
important
important
is
is
that
that
the
the
composite
composite
string
string
is
is
created
created
in
in
the
the
same
same
order
order
that
that
the
the
parent
parent
elements
elements
appear
appear
in
in
the
the
tree.
tree
Let's
let's
take
take
a
a
look
look
at
at
how
how
we
we
can
can
actually
actually
traverse
traverse
this
this
tree.
tree
If
if
we
we
just
just
wanna
wanna
grab
grab
the
the
top
top
level
level
of
of
the
the
tree,
tree
which
which
you
you
would
would
probably
probably
wanna
wanna
do
do
if
if
you're
you're
showing
showing
the
the
top
top
level
level
comments,
comments
is
is
just
just
query
query
where
where
the
the
parent
parent
equals
equals
falls.
falls
Now
now
if
if
you
you
want
want
to
to
query
query
across
across
the
the
breadth
breadth
of
of
the
the
tree,
tree
or
or
just
just
get
get
all
all
of
of
the
the
top
top
level
level
responses
responses
or
or
something
something
like
like
that,
that
you
you
can
can
also
also
save
save
a
a
level
level
property
property
on
on
each
each
document,
document
which
which
would
would
allow
allow
you
you
to
to
do
do
so.
so
But
but
I
i
think
think
most
most
use
use
cases
cases
will
will
require
require
you
you
to
to
query
query
the
the
depth
depth
of
of
the
the
tree,
tree
so
so
this
this
part's
part's
technically
technically
optional.
optional
Then
then
assuming
assuming
you
you
have
have
the
the
level
level
property,
property
which
which
is
is
just
just
an
an
integer
integer
that
that
increases
increases
by
by
1
1
for
for
each
each
to
to
level,
level
you
you
can
can
use
use
where
where
to
to
query
query
where
where
all
all
the
the
documents
documents
live
live
at
at
a
a
certain
certain
level,
level
or
or
you
you
could
could
use
use
a
a
range
range
operator
operator
to
to
get
get
everything
everything
above
above
or
or
below
below
a
a
certain
certain
level.
level
Now
now
things
things
get
get
a
a
lot
lot
more
more
interesting
interesting
when
when
we
we
query
query
the
the
depth
depth
of
of
the
the
tree.
tree
The
the
composite
composite
key
key
that
that
we
we
set
set
up
up
earlier
earlier
will
will
get
get
larger
larger
and
and
larger
larger
as
as
we
we
get
get
deeper
deeper
in
in
the
the
tree.
tree
That
that
means
means
we
we
can
can
take
take
a
a
document
document
ID
id
and
and
use
use
it
it
as
as
a
a
starting
starting
point
point
and
and
then
then
query
query
all
all
of
of
the
the
children
children
that
that
start
start
with
with
that
that
same
same
ID
id
in
in
their
their
composite
composite
parent
parent
string.
string
We
we
can
can
make
make
this
this
query
query
by
by
saying
saying
where
where
the
the
parent
parent
is
is
greater
greater
than
than
or
or
equal
equal
to
to
the
the
ID
id
and
and
where
where
the
the
parent
parent
is
is
less
less
than
than
or
or
equal
equal
to
to
the
the
ID
id
plus
plus
a
a
high
high
Unicode
unicode
character.
character
So
so
that
that
would
would
allow
allow
us
us
to
to
start
start
from
from
any
any
node
node
in
in
the
the
tree
tree
and
and
then
then
traverse
traverse
downward.
downward
And
and
this
this
is
is
actually
actually
the
the
same
same
way
way
that
that
geohashing
geohashing
works,
works
and
and
it's
it's
a
a
very
very
powerful
powerful
feature
feature
but
but
is
is
a
a
little
little
more
more
advanced.
advanced
Now
now
moving
moving
on
on
to
to
our
our
next
next
data
data
model,
model
which
which
comes
comes
from
from
Stephan.
stephan
We're
we're
going
going
to
to
look
look
at
at
how
how
we
we
can
can
query
query
a
a
collection
collection
starting
starting
with
with
an
an
array
array
of
of
document
document
IDs.
ids
His
his
data
data
model
model
was
was
a
a
lot
lot
more
more
complex,
complex
but
but
I
i
just
just
wanted
wanted
to
to
pick
pick
out
out
one
one
little
little
thing
thing
that
that
I
i
think
think
is
is
helpful
helpful
to
to
a
a
lot
lot
of
of
people.
people
When
when
working
working
with
with
Firestore,
firestore
it's
it's
ideal
ideal
to
to
denormalize
denormalize
your
your
data
data
or
or
embed
embed
it
it
on
on
single
single
document,
document
but
but
there
there
are
are
a
a
lot
lot
of
of
cases
cases
when
when
it's
it's
just
just
not
not
practical
practical
or
or
possible
possible
to
to
do
do
that.
that
So
so
one
one
thing
thing
you
you
can
can
do
do
is
is
create
create
a
a
more
more
normalized
normalized
model
model
like
like
we
we
see
see
here
here
with
with
a
a
sizes
sizes
array,
array
in
in
each
each
element
element
in
in
that
that
array
array
is
is
a
a
string
string
referencing
referencing
a
a
document
document
in
in
a
a
different
different
collection.
collection
But
but
because
because
there
there
are
are
no
no
server
server
side
side
joins
joins
in
in
Firestore,
firestore
how
how
do
do
we
we
actually
actually
join
join
this
this
data
data
so
so
we
we
can
can
use
use
it
it
in
in
our
our
UI?
ui
We
we
we
we
can
can
actually
actually
do
do
this
this
very
very
efficiently
efficiently
in
in
Firestore
firestore
because
because
we
we
can
can
send
send
multiple
multiple
read
read
requests
requests
to
to
the
the
database,
database
and
and
Firestore
firestore
will
will
handle
handle
all
all
of
of
those
those
requests
requests
concurrently.
concurrently
Here's
here's
a
a
little
little
helper
helper
function
function
you
you
can
can
use
use
in
in
JavaScript
javascript
to
to
join
join
an
an
array
array
of
of
IDs
ids
to
to
a
a
collection.
collection
It
it
takes
takes
a
a
collection
collection
and
and
an
an
array
array
of
of
IDs
ids
as
as
its
its
argument,
argument
then
then
it
it
maps
maps
the
the
IDs
ids
to
to
a
a
document
document
read.
read
That
that
will
will
give
give
us
us
an
an
array
array
of
of
promises,
promises
and
and
then
then
we
we
can
can
use
use
promise
promise
dot
dot
all
all
to
to
resolve
resolve
all
all
of
of
those
those
promises
promises
concurrently.
concurrently
And
and
lastly,
lastly
we
we
can
can
map
map
the
the
document
document
snapshot
snapshot
to
to
its
its
raw
raw
data.
data
Now
now
if
if
we
we
wanted
wanted
to
to
use
use
this
this
helper
helper
method
method
in
in
our
our
code,
code
we
we
can
can
just
just
pass
pass
it
it
this
this
collection
collection
reference
reference
and
and
the
the
array
array
of
of
IDs
ids
that
that
we
we
wanna
wanna
read,
read
and
and
it
it
will
will
return
return
an
an
array
array
of
of
the
the
document
document
data.
data
So
so
if
if
you
you
want
want
to
to
model
model
your
your
data
data
in
in
a
a
more
more
normalized
normalized
SQL
sql
style,
style
you
you
can
can
do
do
it
it
using
using
something
something
like
like
this.
this
Now
now
that
that
brings
brings
us
us
to
to
the
the
final
final
model
model
coming
coming
from
from
Troy,
troy
and
and
he's
he's
building
building
something
something
like
like
a
a
social
social
media
media
style
style
follower
follower
feed.
feed
In
in
other
other
words,
words
you
you
follow
follow
a
a
bunch
bunch
of
of
users
users
and
and
you
you
wanna
wanna
see
see
the
the
most
most
recent
recent
posts
posts
from
from
those
those
users
users
that
that
you
you
follow.
follow
This
this
is
is
actually
actually
a
a
very
very
challenging
challenging
requirement,
requirement
so
so
I
i
built
built
a
a
full
full
demo
demo
to
to
show
show
that
that
it's
it's
possible
possible
in
in
Firestore.
firestore
As
as
you
you
can
can
see
see
here,
here
I'm
i'm
logged
logged
in
in
as
as
Jeff
jeff
d
d
23.
23
And
and
when
when
I
i
unfollow
unfollow
these
these
users,
users
their
their
posts
posts
are
are
removed
removed
from
from
my
my
feed.
feed
So,
so
basically,
basically
what
what
we're
we're
doing
doing
here
here
is
is
we're
we're
having
having
the
the
most
most
recent
recent
posts
posts
from
from
these
these
users
users
and
and
then
then
ordering
ordering
them
them
by
by
date.
date
And
and
on
on
top
top
of
of
that,
that
we're
we're
also
also
maintaining
maintaining
the
the
user
user
to
to
user
user
follow
follow
relationship.
relationship
Let's
let's
start
start
by
by
taking
taking
a
a
look
look
at
at
the
the
data
data
model.
model
The
the
only
only
way
way
to
to
make
make
a
a
system
system
like
like
this
this
scale
scale
to
to
a
a
decent
decent
amount
amount
of
of
users
users
is
is
to
to
duplicate
duplicate
some
some
data.
data
In
in
our
our
database,
database
we
we
have
have
3
3
collections
collections
at
at
play.
play
We
we
have
have
one
one
for
for
users,
users
one
one
for
for
posts
posts
that
that
can
can
be
be
posted
posted
by
by
a
a
user
user
and
and
then
then
another
another
one
one
for
for
followers.
followers
All
all
the
the
relational
relational
data
data
will
will
be
be
handled
handled
by
by
the
the
followers
followers
collection.
collection
And
and
in
in
fact,
fact
all
all
of
of
the
the
data
data
that
that
you
you
see
see
in
in
the
the
UI
ui
is
is
coming
coming
from
from
a
a
query
query
from
from
this
this
followers
followers
document.
document
The
the
followers
followers
document
document
belongs
belongs
to
to
the
the
user
user
who
who
is
is
being
being
followed
followed
and
and
keeps
keeps
track
track
of
of
all
all
of
of
the
the
followers
followers
was
was
in
in
an
an
array.
array
But
but
in
in
addition
addition
to
to
keeping
keeping
track
track
of
of
the
the
followers,
followers
it
it
also
also
duplicates
duplicates
some
some
of
of
the
the
user's
user's
recent
recent
posts.
posts
It
it
doesn't
doesn't
duplicate
duplicate
everything.
everything
It
it
only
only
duplicates
duplicates
the
the
data
data
that
that
we
we
need
need
is
is
showing
showing
the
the
feed,
feed
like
like
the
the
title
title
and
and
then
then
maybe
maybe
a
a
preview
preview
of
of
some
some
of
of
the
the
text.
text
And
and
the
the
last
last
thing
thing
we
we
need
need
to
to
do
do
is
is
keep
keep
a
a
time
time
stamp
stamp
of
of
the
the
user's
user's
last
last
post.
post
What
what
we're
we're
going
going
to
to
do
do
is
is
make
make
a
a
query
query
on
on
this
this
last
last
post
post
property
property
along
along
with
with
the
the
user's
user's
array,
array
and
and
the
the
result
result
of
of
that
that
will
will
be
be
all
all
of
of
these
these
duplicated
duplicated
posts
posts
from
from
the
the
users
users
that
that
the
the
user
user
is
is
following.
following
Now
now
an
an
important
important
thing
thing
to
to
point
point
out
out
here
here
is
is
that
that
we're
we're
keeping
keeping
track
track
of
of
the
the
followers
followers
on
on
a
a
single
single
document,
document
so
so
that
that
means
means
we
we
can
can
only
only
scale
scale
up
up
to
to
1
1
megabyte
megabyte
of
of
data.
data
And
and
that
that
means
means
you
you
may
may
need
need
to
to
break
break
this
this
up
up
into
into
multiple
multiple
documents
documents
as
as
your
your
app
app
grows.
grows
I'm
i'm
going
going
to
to
go
go
ahead
ahead
and
and
write
write
this
this
query
query
inside
inside
of
of
an
an
async
async
function
function
because
because
we'll
we'll
need
need
to
to
do
do
some
some
data
data
wrangling
wrangling
after
after
we
we
retrieve
retrieve
the
the
initial
initial
data.
data
First,
first
we'll
we'll
make
make
a
a
reference
reference
to
to
the
the
followers
followers
collection
collection
and
and
then
then
we'll
we'll
query
query
it
it
using
using
array
array
contains
contains
with
with
the
the
username
username
of
of
the
the
logged
logged
in
in
user.
user
After
after
that,
that
we'll
we'll
make
make
this
this
a
a
compound
compound
query
query
by
by
using
using
order
order
by
by
with
with
the
the
last
last
post
post
time
time
stamp.
stamp
And
and
keep
keep
in
in
mind
mind
this
this
is
is
a
a
compound
compound
query,
query
so
so
it's
it's
going
going
to
to
require
require
an
an
index,
index
and
and
Firestore
firestore
will
will
give
give
you
you
a
a
warning
warning
about
about
that
that
in
in
the
the
console.
console
Now
now
the
the
cool
cool
thing
thing
about
about
this
this
is
is
we
we
can
can
limit
limit
it
it
to
to
10
10
document
document
reads,
reads
so
so
it's
it's
a
a
very
very
efficient
efficient
read,
read
but
but
it
it
can
can
provide
provide
us
us
with
with
data
data
to
to
populate
populate
potentially
potentially
dozens
dozens
of
of
different
different
posts
posts
in
in
the
the
feed.
feed
Once
once
we
we
have
have
the
the
result
result
of
of
this
this
query,
query
we
we
have
have
an
an
array
array
that
that
we
we
need
need
to
to
then
then
map
map
to
to
the
the
document
document
data.
data
And
and
then
then
on
on
the
the
document
document
data,
data
we're
we're
going
going
to
to
have
have
an
an
array
array
of
of
the
the
user's
user's
most
most
recent
recent
posts,
posts
and
and
that's
that's
what
what
we
we
actually
actually
want
want
to
to
be
be
showing
showing
in
in
the
the
view.
view
So
so
I'm
i'm
going
going
to
to
reduce
reduce
this
this
array
array
of
of
documents
documents
down
down
to
to
a
a
new
new
array
array
that
that
only
only
contains
contains
the
the
recent
recent
posts,
posts
and
and
then
then
I
i
want
want
to
to
sort
sort
those
those
recent
recent
posts
posts
based
based
on
on
their
their
publish
publish
date.
date
We
we
can
can
use
use
array
array
sort
sort
to
to
handle
handle
that.
that
And
and
at
at
this
this
point,
point
we
we
now
now
have
have
a
a
sorted
sorted
array
array
of
of
posts
posts
that
that
spans
spans
across
across
multiple
multiple
users
users
that
that
this
this
user
user
is
is
following.
following
I'm
i'm
I'm
i'm
gonna
gonna
go
go
ahead
ahead
and
and
wrap
wrap
things
things
up
up
there.
there
If
if
you
you
wanna
wanna
learn
learn
more
more
about
about
data
data
modeling
modeling
and
and
how
how
these
these
particular
particular
data
data
models
models
work,
work
consider
consider
becoming
becoming
a
a
pro
pro
member
member
at
at
Fireship
fireship
IO
io
to
to
get
get
access
access
to
to
the
the
course.
course
Thanks
thanks
for
for
watching,
watching
and
and
I
i
will
will
talk
talk
to
to
you
you
soon.
soon
success
The speakers discuss optimal data structures for apps, including optimal No Guinea database structures and optimal data models. They also talk about creating a composite key and filtering through multiple queries to retrieve all of the data in a single file. They explain how Firwork works to create a composite key and how it can be used to maintain a user to user follow relationship. Finally, they discuss creating a query for a list of posts and how it can be used to maintain a user to user follow relationship.
Firestore Data Modeling - Five Cool Techniques
One of the most challenging aspects of building any app is determining the optimal data structure. Ideally, you want to maximize performance while minimizing costs. But that's easier said than done, and especially so with a NoSQL database like Firestore, because you really have to plan ahead on how you model your data. Last week, I asked everybody on our Slack channel to send me the biggest challenges that they faced modeling data in Firestore, store, and this conversation snowballed into an entire new course on Fireship dedicated completely to NoSQL data modeling and queries. So if this is a topic you're interested in, consider becoming a Pro Member to get access to that course, and I'll leave a discount code in the video description. In today's video, we're going to look at 5 different ways you can structure and query your data that you may not know about. And each technique that we'll look at is based on a real submission from the community. A minute ago I said that you have to plan ahead when structuring your data. So what do I mean by that exactly? Well, the basic idea is that you want to pre render your data so that it fits the view or the screen in your app as closely as possible. Ideally, you only need to make one document read or one query to a collection to fill the entire view with data. This differs from a relational SQL database where the idea is to break your data into a lot small pieces, and then join it all together server side. Now I've heard people say that Firestore or NoSQL databases in general are not good for relational data modeling, but that's actually not true, and it's really just a matter of rethinking the way that you model data in general. And what you'll find is that you can solve all the same problems that you can with SQL, but you can do it in a way is faster, less expensive, and more flexible. With that being said, let's go ahead and jump into our first data model and access pattern. This suggestion comes from Somrat, and he wants to query a collection based on whether or not a user was mentioned in a post in that collection. This is similar to something like Twitter where if you mention a username the at symbol, it will notify that user that they've been mentioned, and they also might want to see a list of all the tweets that they've been mentioned in. In this case, we have a many to many relationship where a tweet can have many mentioned users, and a user can be mentioned in many tweets. But the thing to keep in mind here is that a tweet or a post can maybe only have 1 to 10 mentioned users, but a user can be mentioned in potentially billions of tweets. In situations like this, it's usually best to have the document with the smaller side of the relational data manage the relationships. In in this example, we have an array of usernames embedded on the tweet or post document. You'd implement this logic somewhere in your front end code using regex is to extract the usernames out of the post text and then duplicate them on the same document as an array. Now it becomes very easy to query all the tweets that a user is mentioned in by making a reference to the post collection, then we can use the array contains operator to get all the tweets that have a corresponding user ID in that mentions array. Then we could use a cloud function to listen for a new post to be created and then send a push notification or an email to your user when they're mentioned. So the main limitation what array contains is that you can only query for 1 item at a time. And that leads me to the next challenge, which is a category system that allows you to filter by multiple categories at the same time while also being able to see if a post contains a given category out of potentially multiple past values and also the ability to filter by not containing a certain value. So these requirements roughly translate into an and or or not statement in a SQL database. I'm gonna go ahead and call these tags instead of categories, but the general idea here is the same. We have a tags collection, which is really just there to hold any extra data about a tag, like a description or a URL or something like that. And I'd recommend giving your tags a unique ID that's something descriptive that you can actually show in the UI. And then you'll take that ID and associate it to a post by using a map. So we have a map called tags, and then each key in that map is the ID from the tags collection. And the cool thing about Firestore is that it automatically indexes the keys in the map, which means we can query them without any additional configuration. First, let's take a look at how we can do a logical inquiry, or in other words, we want to get all of the items that have both of these tags present. We can do this by simply chaining together where statements. And as long as they're using the equal to operator, we can do this as many times as we want. So this works great if you're filtering by multiple Boolean values. But if you throw in a range operator, that means you'll need a composite index. And keep in mind, you can only use 1 range operator per query, so that's a limitation that you'll want to consider. Now if you wanted to make something that was more like an or query, you can just make multiple queries for separate tags. We're able to run these queries concurrently, and then we can just join them together and filter throughout any duplicates in our client side code. Now the most difficult part of this challenge is implementing the not logic. The values on the map are set up as booleans, so the most straightforward approach here would be to add all of the tags to every document and then set them to false by default. This would be perfectly fine if you have a finite set of tags and know their values up front, but it would be much more challenging if tags are generated by users and there could be potentially billions of tags. So that's a limitation to consider. But keep in mind that if you have a numeric value or if you have a string value with some sort of ordering embedded in it, then you can simulate a knot query by using range operators. For example, if we wanted to get all of the items that were not $20, then we could query on either side of that range. Or in other words, all the items that are less than $20 and all the items that are more than $20. And if you're doing a lot of stuff like this, it might be time to look into something like Algolia to index some of your data in a full text search engine. That's what I do for Fireship, and Firebase and Algolia work really well together. Now moving on to our next data model, this one comes from Lenny Cunningham and his flight now dot net app. Geolocation queries bring up a really interesting data modeling technique called composite strings. Not only do composite strings allow us to do things like geohashing, but it also allows us to do things like tree traversals and threaded comments. Take a look at this tree structure we have here where each letter represents a document in the database. What I'm going to show you next is how to write a query that will traverse down one node of the tree. And and this is especially useful if you're building something like Reddit comments that can be threaded multiple levels deep, or if you have a hierarchy of categories and only want to query a specific node in that tree. We can do this with all of the documents in the same collection. And you can see here at the top of the tree, we have the document a, and it has a parent value set to false. That would be our top level comment. And then let's say a user responds to this comment, that will be our b document, and it has a parent value set to a. Now let's imagine we have a response to the b document, then we would have a c and d document, and they both reference the same parents of a, b. So So what we've done here is create a composite key where the items at the 3rd level of the tree reference the IDs of the parent documents at the first and second levels. These don't need to be an alphabetical order or anything like that. You can just use Firestore's automatically generated IDs. The only thing that's important is that the composite string is created in the same order that the parent elements appear in the tree. Let's take a look at how we can actually traverse this tree. If we just wanna grab the top level of the tree, which you would probably wanna do if you're showing the top level comments, is just query where the parent equals falls. Now if you want to query across the breadth of the tree, or just get all of the top level responses or something like that, you can also save a level property on each document, which would allow you to do so. But I think most use cases will require you to query the depth of the tree, so this part's technically optional. Then assuming you have the level property, which is just an integer that increases by 1 for each to level, you can use where to query where all the documents live at a certain level, or you could use a range operator to get everything above or below a certain level. Now things get a lot more interesting when we query the depth of the tree. The composite key that we set up earlier will get larger and larger as we get deeper in the tree. That means we can take a document ID and use it as a starting point and then query all of the children that start with that same ID in their composite parent string. We can make this query by saying where the parent is greater than or equal to the ID and where the parent is less than or equal to the ID plus a high Unicode character. So that would allow us to start from any node in the tree and then traverse downward. And this is actually the same way that geohashing works, and it's a very powerful feature but is a little more advanced. Now moving on to our next data model, which comes from Stephan. We're going to look at how we can query a collection starting with an array of document IDs. His data model was a lot more complex, but I just wanted to pick out one little thing that I think is helpful to a lot of people. When working with Firestore, it's ideal to denormalize your data or embed it on single document, but there are a lot of cases when it's just not practical or possible to do that. So one thing you can do is create a more normalized model like we see here with a sizes array, in each element in that array is a string referencing a document in a different collection. But because there are no server side joins in Firestore, how do we actually join this data so we can use it in our UI? We we can actually do this very efficiently in Firestore because we can send multiple read requests to the database, and Firestore will handle all of those requests concurrently. Here's a little helper function you can use in JavaScript to join an array of IDs to a collection. It takes a collection and an array of IDs as its argument, then it maps the IDs to a document read. That will give us an array of promises, and then we can use promise dot all to resolve all of those promises concurrently. And lastly, we can map the document snapshot to its raw data. Now if we wanted to use this helper method in our code, we can just pass it this collection reference and the array of IDs that we wanna read, and it will return an array of the document data. So if you want to model your data in a more normalized SQL style, you can do it using something like this. Now that brings us to the final model coming from Troy, and he's building something like a social media style follower feed. In other words, you follow a bunch of users and you wanna see the most recent posts from those users that you follow. This is actually a very challenging requirement, so I built a full demo to show that it's possible in Firestore. As you can see here, I'm logged in as Jeff d 23. And when I unfollow these users, their posts are removed from my feed. So, basically, what we're doing here is we're having the most recent posts from these users and then ordering them by date. And on top of that, we're also maintaining the user to user follow relationship. Let's start by taking a look at the data model. The only way to make a system like this scale to a decent amount of users is to duplicate some data. In our database, we have 3 collections at play. We have one for users, one for posts that can be posted by a user and then another one for followers. All the relational data will be handled by the followers collection. And in fact, all of the data that you see in the UI is coming from a query from this followers document. The followers document belongs to the user who is being followed and keeps track of all of the followers was in an array. But in addition to keeping track of the followers, it also duplicates some of the user's recent posts. It doesn't duplicate everything. It only duplicates the data that we need is showing the feed, like the title and then maybe a preview of some of the text. And the last thing we need to do is keep a time stamp of the user's last post. What we're going to do is make a query on this last post property along with the user's array, and the result of that will be all of these duplicated posts from the users that the user is following. Now an important thing to point out here is that we're keeping track of the followers on a single document, so that means we can only scale up to 1 megabyte of data. And that means you may need to break this up into multiple documents as your app grows. I'm going to go ahead and write this query inside of an async function because we'll need to do some data wrangling after we retrieve the initial data. First, we'll make a reference to the followers collection and then we'll query it using array contains with the username of the logged in user. After that, we'll make this a compound query by using order by with the last post time stamp. And keep in mind this is a compound query, so it's going to require an index, and Firestore will give you a warning about that in the console. Now the cool thing about this is we can limit it to 10 document reads, so it's a very efficient read, but it can provide us with data to populate potentially dozens of different posts in the feed. Once we have the result of this query, we have an array that we need to then map to the document data. And then on the document data, we're going to have an array of the user's most recent posts, and that's what we actually want to be showing in the view. So I'm going to reduce this array of documents down to a new array that only contains the recent posts, and then I want to sort those recent posts based on their publish date. We can use array sort to handle that. And at this point, we now have a sorted array of posts that spans across multiple users that this user is following. I'm I'm gonna go ahead and wrap things up there. If you wanna learn more about data modeling and how these particular data models work, consider becoming a pro member at Fireship IO to get access to the course. Thanks for watching, and I will talk to you soon.
The speakers discuss optimal data structures for apps, including optimal No Guinea database structures and optimal data models. They also talk about creating a composite key and filtering through multiple queries to retrieve all of the data in a single file. They explain how Firwork works to create a composite key and how it can be used to maintain a user to user follow relationship. Finally, they discuss creating a query for a list of posts and how it can be used to maintain a user to user follow relationship.

        [INST]
        You are youtuber called Fireship you make engaging high-intensity and entertaining coding tutorials and tech news. 
        you covers a wide range of topics relevant to programmers, aiming to help them learn and improve their skills quickly.
        
        Given the title of the video : Firestore Data Modeling - Five Cool Techniques 
        and a small summary : The speakers discuss optimal data structures for apps, including optimal No Guinea database structures and optimal data models. They also talk about creating a composite key and filtering through multiple queries to retrieve all of the data in a single file. They explain how Firwork works to create a composite key and how it can be used to maintain a user to user follow relationship. Finally, they discuss creating a query for a list of posts and how it can be used to maintain a user to user follow relationship.
        [/INST]
        
        Generate the video : One of the most challenging aspects of building any app is determining the optimal data structure. Ideally, you want to maximize performance while minimizing costs. But that's easier said than done, and especially so with a NoSQL database like Firestore, because you really have to plan ahead on how you model your data. Last week, I asked everybody on our Slack channel to send me the biggest challenges that they faced modeling data in Firestore, store, and this conversation snowballed into an entire new course on Fireship dedicated completely to NoSQL data modeling and queries. So if this is a topic you're interested in, consider becoming a Pro Member to get access to that course, and I'll leave a discount code in the video description. In today's video, we're going to look at 5 different ways you can structure and query your data that you may not know about. And each technique that we'll look at is based on a real submission from the community. A minute ago I said that you have to plan ahead when structuring your data. So what do I mean by that exactly? Well, the basic idea is that you want to pre render your data so that it fits the view or the screen in your app as closely as possible. Ideally, you only need to make one document read or one query to a collection to fill the entire view with data. This differs from a relational SQL database where the idea is to break your data into a lot small pieces, and then join it all together server side. Now I've heard people say that Firestore or NoSQL databases in general are not good for relational data modeling, but that's actually not true, and it's really just a matter of rethinking the way that you model data in general. And what you'll find is that you can solve all the same problems that you can with SQL, but you can do it in a way is faster, less expensive, and more flexible. With that being said, let's go ahead and jump into our first data model and access pattern. This suggestion comes from Somrat, and he wants to query a collection based on whether or not a user was mentioned in a post in that collection. This is similar to something like Twitter where if you mention a username the at symbol, it will notify that user that they've been mentioned, and they also might want to see a list of all the tweets that they've been mentioned in. In this case, we have a many to many relationship where a tweet can have many mentioned users, and a user can be mentioned in many tweets. But the thing to keep in mind here is that a tweet or a post can maybe only have 1 to 10 mentioned users, but a user can be mentioned in potentially billions of tweets. In situations like this, it's usually best to have the document with the smaller side of the relational data manage the relationships. In in this example, we have an array of usernames embedded on the tweet or post document. You'd implement this logic somewhere in your front end code using regex is to extract the usernames out of the post text and then duplicate them on the same document as an array. Now it becomes very easy to query all the tweets that a user is mentioned in by making a reference to the post collection, then we can use the array contains operator to get all the tweets that have a corresponding user ID in that mentions array. Then we could use a cloud function to listen for a new post to be created and then send a push notification or an email to your user when they're mentioned. So the main limitation what array contains is that you can only query for 1 item at a time. And that leads me to the next challenge, which is a category system that allows you to filter by multiple categories at the same time while also being able to see if a post contains a given category out of potentially multiple past values and also the ability to filter by not containing a certain value. So these requirements roughly translate into an and or or not statement in a SQL database. I'm gonna go ahead and call these tags instead of categories, but the general idea here is the same. We have a tags collection, which is really just there to hold any extra data about a tag, like a description or a URL or something like that. And I'd recommend giving your tags a unique ID that's something descriptive that you can actually show in the UI. And then you'll take that ID and associate it to a post by using a map. So we have a map called tags, and then each key in that map is the ID from the tags collection. And the cool thing about Firestore is that it automatically indexes the keys in the map, which means we can query them without any additional configuration. First, let's take a look at how we can do a logical inquiry, or in other words, we want to get all of the items that have both of these tags present. We can do this by simply chaining together where statements. And as long as they're using the equal to operator, we can do this as many times as we want. So this works great if you're filtering by multiple Boolean values. But if you throw in a range operator, that means you'll need a composite index. And keep in mind, you can only use 1 range operator per query, so that's a limitation that you'll want to consider. Now if you wanted to make something that was more like an or query, you can just make multiple queries for separate tags. We're able to run these queries concurrently, and then we can just join them together and filter throughout any duplicates in our client side code. Now the most difficult part of this challenge is implementing the not logic. The values on the map are set up as booleans, so the most straightforward approach here would be to add all of the tags to every document and then set them to false by default. This would be perfectly fine if you have a finite set of tags and know their values up front, but it would be much more challenging if tags are generated by users and there could be potentially billions of tags. So that's a limitation to consider. But keep in mind that if you have a numeric value or if you have a string value with some sort of ordering embedded in it, then you can simulate a knot query by using range operators. For example, if we wanted to get all of the items that were not $20, then we could query on either side of that range. Or in other words, all the items that are less than $20 and all the items that are more than $20. And if you're doing a lot of stuff like this, it might be time to look into something like Algolia to index some of your data in a full text search engine. That's what I do for Fireship, and Firebase and Algolia work really well together. Now moving on to our next data model, this one comes from Lenny Cunningham and his flight now dot net app. Geolocation queries bring up a really interesting data modeling technique called composite strings. Not only do composite strings allow us to do things like geohashing, but it also allows us to do things like tree traversals and threaded comments. Take a look at this tree structure we have here where each letter represents a document in the database. What I'm going to show you next is how to write a query that will traverse down one node of the tree. And and this is especially useful if you're building something like Reddit comments that can be threaded multiple levels deep, or if you have a hierarchy of categories and only want to query a specific node in that tree. We can do this with all of the documents in the same collection. And you can see here at the top of the tree, we have the document a, and it has a parent value set to false. That would be our top level comment. And then let's say a user responds to this comment, that will be our b document, and it has a parent value set to a. Now let's imagine we have a response to the b document, then we would have a c and d document, and they both reference the same parents of a, b. So So what we've done here is create a composite key where the items at the 3rd level of the tree reference the IDs of the parent documents at the first and second levels. These don't need to be an alphabetical order or anything like that. You can just use Firestore's automatically generated IDs. The only thing that's important is that the composite string is created in the same order that the parent elements appear in the tree. Let's take a look at how we can actually traverse this tree. If we just wanna grab the top level of the tree, which you would probably wanna do if you're showing the top level comments, is just query where the parent equals falls. Now if you want to query across the breadth of the tree, or just get all of the top level responses or something like that, you can also save a level property on each document, which would allow you to do so. But I think most use cases will require you to query the depth of the tree, so this part's technically optional. Then assuming you have the level property, which is just an integer that increases by 1 for each to level, you can use where to query where all the documents live at a certain level, or you could use a range operator to get everything above or below a certain level. Now things get a lot more interesting when we query the depth of the tree. The composite key that we set up earlier will get larger and larger as we get deeper in the tree. That means we can take a document ID and use it as a starting point and then query all of the children that start with that same ID in their composite parent string. We can make this query by saying where the parent is greater than or equal to the ID and where the parent is less than or equal to the ID plus a high Unicode character. So that would allow us to start from any node in the tree and then traverse downward. And this is actually the same way that geohashing works, and it's a very powerful feature but is a little more advanced. Now moving on to our next data model, which comes from Stephan. We're going to look at how we can query a collection starting with an array of document IDs. His data model was a lot more complex, but I just wanted to pick out one little thing that I think is helpful to a lot of people. When working with Firestore, it's ideal to denormalize your data or embed it on single document, but there are a lot of cases when it's just not practical or possible to do that. So one thing you can do is create a more normalized model like we see here with a sizes array, in each element in that array is a string referencing a document in a different collection. But because there are no server side joins in Firestore, how do we actually join this data so we can use it in our UI? We we can actually do this very efficiently in Firestore because we can send multiple read requests to the database, and Firestore will handle all of those requests concurrently. Here's a little helper function you can use in JavaScript to join an array of IDs to a collection. It takes a collection and an array of IDs as its argument, then it maps the IDs to a document read. That will give us an array of promises, and then we can use promise dot all to resolve all of those promises concurrently. And lastly, we can map the document snapshot to its raw data. Now if we wanted to use this helper method in our code, we can just pass it this collection reference and the array of IDs that we wanna read, and it will return an array of the document data. So if you want to model your data in a more normalized SQL style, you can do it using something like this. Now that brings us to the final model coming from Troy, and he's building something like a social media style follower feed. In other words, you follow a bunch of users and you wanna see the most recent posts from those users that you follow. This is actually a very challenging requirement, so I built a full demo to show that it's possible in Firestore. As you can see here, I'm logged in as Jeff d 23. And when I unfollow these users, their posts are removed from my feed. So, basically, what we're doing here is we're having the most recent posts from these users and then ordering them by date. And on top of that, we're also maintaining the user to user follow relationship. Let's start by taking a look at the data model. The only way to make a system like this scale to a decent amount of users is to duplicate some data. In our database, we have 3 collections at play. We have one for users, one for posts that can be posted by a user and then another one for followers. All the relational data will be handled by the followers collection. And in fact, all of the data that you see in the UI is coming from a query from this followers document. The followers document belongs to the user who is being followed and keeps track of all of the followers was in an array. But in addition to keeping track of the followers, it also duplicates some of the user's recent posts. It doesn't duplicate everything. It only duplicates the data that we need is showing the feed, like the title and then maybe a preview of some of the text. And the last thing we need to do is keep a time stamp of the user's last post. What we're going to do is make a query on this last post property along with the user's array, and the result of that will be all of these duplicated posts from the users that the user is following. Now an important thing to point out here is that we're keeping track of the followers on a single document, so that means we can only scale up to 1 megabyte of data. And that means you may need to break this up into multiple documents as your app grows. I'm going to go ahead and write this query inside of an async function because we'll need to do some data wrangling after we retrieve the initial data. First, we'll make a reference to the followers collection and then we'll query it using array contains with the username of the logged in user. After that, we'll make this a compound query by using order by with the last post time stamp. And keep in mind this is a compound query, so it's going to require an index, and Firestore will give you a warning about that in the console. Now the cool thing about this is we can limit it to 10 document reads, so it's a very efficient read, but it can provide us with data to populate potentially dozens of different posts in the feed. Once we have the result of this query, we have an array that we need to then map to the document data. And then on the document data, we're going to have an array of the user's most recent posts, and that's what we actually want to be showing in the view. So I'm going to reduce this array of documents down to a new array that only contains the recent posts, and then I want to sort those recent posts based on their publish date. We can use array sort to handle that. And at this point, we now have a sorted array of posts that spans across multiple users that this user is following. I'm I'm gonna go ahead and wrap things up there. If you wanna learn more about data modeling and how these particular data models work, consider becoming a pro member at Fireship IO to get access to the course. Thanks for watching, and I will talk to you soon.
        