Doesn't Safari cache pages? Whenever I go back or forward in the Safari navigation history, it reloads the pages even when I just visited the page two seconds ago.

When navigating using back/forward buttons or swiping back and forth, the app always reloads the page from the internet.

Can I change anything to make Safari cache the pages I just visited more aggressively?
The answer is no, Safari doesn't cache pages - it does however cache elements that make up a page when the rules that the web server associates with each element allows caching.

You can see this on your specific pages by turning on the Develop menu in Safari Preferences:



As you can see from this picture, the page where your question is posted allows many items to be cached and the load time for those elements is in the 0.1-0.3ms range. The items that are not cached have longer load durations so perhaps the pages you are navigating either have few items with cache enabled or the overall page load time is dominated by non-cached layout and content.

If you want to cache content for offline reading, use the Reading List feature to store the page for offline use. That does more aggressively cache much of the most pages to allow reading of the page while totally offline at a later date. In your case, when Safari is online, it will check to see that the reading list version of the page is current, so it might not be exactly what you are seeking.