It is because light travels in more-or-less straight rays.
Let's assume for simplicity that your eye is like a pinhole camera; it has a pinhole in front and a screen at the back. Then an image forms by the light rays that pass through the pinhole.
(from https://commons.wikimedia.org/wiki/File:Pinhole-camera.png)
Consider two points on an object, such as the top and bottom of a tree. If you move an object further, the two points remain at the same distance from one another but are further from the pinhole, and hence the angle that it makes with the pinhole is smaller. But the screen is still the same distance from the pinhole, and hence the image is smaller. (See also this related BBC article.)
As for your question about the vanishing point, it's not clear at all what you are asking. In perspective drawing we have the projection of the scene onto a flat screen through an origin (the eye). Under the projection, consider all the lines that do not pass through the origin. Each of them maps to a line, and any two of them that intersect map to intersecting lines. 530
If we stipulate that any two of them that are parallel actually intersect at a point at infinity, then it is even nicer because then any two lines intersect at a unique point. Also, parallel lines on a horizontal surface would all intersect at the same point at infinity, which after projection maps to a point on the screen, through which the images of all those parallel lines pass through.
Points at infinity do not exist in the Euclidean model of the world, just as parallel lines do not intersect. It is just that if we conceptually add them to the Euclidean model we obtain the projective space that has nice properties, including a meaningful notion of the horizon as the image of some line of points at infinity. This enables us to draw perspective drawings, which is basically to draw the image on the screen given what we know about the lines in the scene.
The surface of the Earth is not flat but slightly curved, and so we cannot see the whole surface of the Earth, and we see a slightly curved horizon. This has nothing to do with the line at infinity in perspective drawings, since that line is still there, just not relevant to the horizon that we see between the earth's surface and the sky. If this is where you got your "2.9 miles" from, then it is simply how far on the surface of the earth you can see, which is of course unrelated to the fact that you can see all the way to the stars.
(from https://en.wikipedia.org/wiki/Horizon#/media/File:Horizons.svg)
In the above diagram, the astronomical horizon corresponds to the line at infinity if you were standing on a really flat surface. The true horizon is what you perceive to be the divide between the sky and ocean since the Earth isn't flat. The visible horizon is what you perceive to be the divide between the sky and land since there are usually lots of things like trees on the land blocking the view of the actual ground surface.
In short, perspective drawings don't work for things that are too far away on the Earth's surface because they assume a perfectly flat ground surface. 204