I receive daily from an external source a very large amount of data (around 250GB with 260 million rows of fixed width text) distributed over 5 text files. I am writing a Java application that should combine a first group of data (files 1-4) with a second group (file 5) based on some business logic.

But accessing/reading multiple times from 250 GB of text files is pretty time-consuming. So I decided to find a more efficient way to process my data. I think to store the data into a database (for example mysql Workbench) and make the processing using the database instead of the text files. This database would be dropped after the processing is done.

Could this approach of using a temporary database improve the performance compared to text files ? Or are there any better suggestions how to design this mass processing ?

*Note: my application has to be run on Windows R2 Server with 32Gb of RAM an Intel Xeon Processor E5645 and 1 TB hard-disk*