Accuracy of Quantum Chemistry ML models I am trying to compare the performance of few Quantum Chemistry property prediction ML models. I was looking at the following table from DOI:
10.1039/c7sc02664a



The problem is that it does not contain any units. A preprint (https://arxiv.org/pdf/1909.00259.pdf, Table 4) suggests that the energy units are Hartree. Among energy properties, HOMO, LUMO, and gap have reasonable values for Hartree unit. However, values reported for U0, U, H, and G look absurd. As a comparison, look at MAE for mean value of each property:



These are in [eV] units (1 Ha = 27.212 eV). In other words, for instance MPNN models MAE for U property is 55.78 [eV] vs 8.25 [eV] for the mean baseline. This does not make sense at all. [k cal/mol] sounds more reasonable but then why half the energies in one unit the other half in other unit, and then why there are no units in a paper published by Royal Society of Chemistry. Seems like I'm missing something. Thanks!
Yes, you're exactly right - multiple papers in chemistry ML drop the units.

There are even comparisons (usually by statistics, ML or comp. sci. researchers) where models are compared by "averaging" errors down a column like that. Of course that's meaningless, since you can't average electron volts or Hartree (energies), Debye (dipole moments), and volume (polarizabilities).

Worse, energies are often computed as atomization energies - so for large molecules, they can be enormous...

In my opinion, a more relevant and meaningful statistic would be the Mean Absolute Percent Error (MAPE) which is unitless and easier to understand (e.g., 1% error? 0.01% error?)

That said, the original QM9 paper gives the units Table 3:
Scientific Data (2014) 1, art. 140022