Question: * What's the limitation?
* Is there a specific volume of data each can handle regardless of disk space?
* When to use what assuming licensing is not a problem?

Answer: 
It depends what you are trying to do and obviously it has to do with cost.

* MySQL and Postgres are very widely used by a huge number of startups because its open source and there is a lot of support out there for people using it
* MSSQL is good if you are using MS programming languages because of the ease to connect and use.
* I have never used oracle but know people use it a lot for data warehouseing so can't have that much of a bad name

All of these will suffer from similar issues when scaling because they are RDBMS databases. They do also have decent ways to get round it and with a decent ORM used in your code then it shouldn't matter what you use.

Pick the one that all the developers are comfortable with
Perhaps one variable which people overlook in these cases is the availability of expert support. Okay, so currently there's an oversupply of people who can help you with db issues, efficiency issues, disaster recovery etc. However this may not be always the case in the future, and it's the applications you use it for that may be the defining issue. Are there people in your organization who have experience in one or more of the relevant databases? (as it happens I believe that someone's who's become proficient in say Oracle, can become fairly competent in Sql\*Server or Mysql in a fairly short space of time) You state it's going to be used for your financial systems - perhaps you really need input from a consultant who's worked on implementing and/or supporting financial systems - for example I understand that Sybase is popular in City type firms. Or perhaps there's an off-the-shelf package that utilises a preferred database? Try and define exactly what your system(s) needs to do first.