Question: I am working on a space exploration and combat game, and I can create galaxies, nebulas, solar systems, and load and unload them procedurally as needed. Meaning the stars that the player can see are exactly where they are. I have a coordinate system set up with meter-precision, meaning it's sufficient to map quite several galaxies, but also tiny missiles.

For now, I created a galaxy with 10.000.000 solar systems, and my SQLite database reached 5gb. It takes ~30 minutes to generate it, and I notice a slow-down as the size of the database increases in loading times. I did a lot of optimization, and while everything is running fine and well now, I seek to reach 200-300m solar systems if possible. The database would be stored locally on the user's drive, rather than downloaded from a server. I use a GlobalX/Y/Z (int64) and LocalX/Y/Z (double) for coordinates, and only the GlobalX/Y/Z is used to find large objects like stars and planets (I also use multiple chunk systems).

[Elite Dangerous manages to store the data of 400 *billion*](https://elite-dangerous.fandom.com/wiki/Galaxy) solar systems and has no issues with loading times, somehow. How can they display many stars in the background? How do they manage to store a lot more data than I already do?

For now, my data is quite minimalistic and comparatively minuscule in number, but they must have quite a lot of data for each stellar object even. What technical "tricks" can I employ to solve this problem in my game?

Answer: 
In addition to the points that have been made about not actually generating all the data there's another factor: You can't actually draw 10,000,000 stars. Nobody has a display that can handle it. Therefore, there's no need to have 10,000,000 stars loaded.

Select X, Y, Brightness, Color, ID from StarTable Where X > CurrentX - 200ly And X < CurrentX + 200ly And Y > CurrentY - 200ly And Y < CurrentY + 200ly.

Select X, Y, Brightness, Color, ID from BrightStarTable Where X > CurrentX - 2000ly And X < CurrentX + 2000ly And Y > CurrentY - 2000ly And Y < CurrentY + 2000ly.

That will be a tiny fraction of your total stars, but it is sufficient to draw anything you can see out the window of your starship. I would store all the details on the stars in a separate table accessed by the StarID as you only need those details when examining a particular star.

You can cut the loaded data down even more by having a DimStarTable with an even shorter query range. Most stars will be in this table.

Another option also comes to mind: Define a new variable: Sector. Sector = (int)(x/100ly) + (int)(y/100ly) \* 10000. You load the sector you are in and the 8 adjoining sectors. The select would run faster. (Likewise, the bright star table would have 1000ly sectors.)
Philipp already gave a great answer, but I'll address your question about how Elite Dangerous operates their database.

> 
> Elite Dangerous manages to store the data of 400 billion solar systems and has no issues with loading times, somehow. How can they display many stars in the background? How do they manage to store a lot more data than I already do?
> 
> 
> 

1. Elite Dangerous stores the star map data that isn't procedurally generated on a server, not locally. They presumably have one or more very powerful servers, optimized for big data, with the database always loaded and running. When the player scrolls around the map or visits a new star system, the game client sends a request to the server, the server queries relevant data from the database and sends it back to the game client. Because the database is on a server, it could potentially be many terabytes in size without causing any issues for the players. They probably have very big and expensive servers developed over years by backend specialists, so this approach is unlikely to be feasible for an indie team.
2. Elite Dangerous absolutely has loading times, they just cleverly mask it with in-game mechanics. This is educated guesswork, but based on my play experience I think it works like this:

	1. When you activate the hyperdrive, there's a progress bar while the hyperdrive warms up. I think this is the stage when the game client sends the request for data for the system you're about to jump to. Occasionally, this takes much longer than normal for no explicable reason - probably because the server was slow to respond to the query.
	2. During the actual hyperspace jump, the game is loading/procedurally generating the next star system, including the background skybox, based on the data it received from the server. The animation you see of your ship hurtling through hyperspace is just a very fancy loading screen.