Should I sub-sample very large datasets to run the Kolmogorov-Smirnov (KS) test? I have two lists with many millions of values (one has 21,410,024,757 values and the other 10,561,427 values). When I run the two-sample KS test the p-value is 0, but many posts suggest that for such large datasets the KS test isn't reliable, e.g:
Kolmogorov-Smirnov test statistic interpretation with large samples
Is it a good approach to sub-sample the two lists of values randomly and uniformly?

Can you say the source of the data and what practical goal you hope to reach doing a K-S test on these two huge samples.
in particular:

*

*With such large datasets, you should be able to see some obvious differences between the the two samples, unless they are essentially from the same distribution,


*Why would a P-value of $\approx 0$ unsatisfactory for your purposes?


*A major complaint about the K-S test is that it sometimes detects differences not of practical importance when sample sizes are very large.
Consider the following, using R:
set.seed(2021)
x1 = rgamma(10^6, 3, .1)
x2 = rgamma(10^5, 3, .1)
x3 = rgamma(10^5, 3, .101)
ks.test(x1, x2)

        Two-sample Kolmogorov-Smirnov test

data:  x1 and x2
D = 0.003331, p-value = 0.2654
alternative hypothesis: two-sided

ks.test(x1, x3)

        Two-sample Kolmogorov-Smirnov test

data:  x1 and x3
D = 0.008329, p-value = 6.656e-06
alternative hypothesis: two-sided

mean(x1); mean(x2); mean(x3)
[1] 30.00744
[1] 30.0714
[1] 29.70087
boxplot(x1,x2,x3, col="skyblue2", horizontal=T)


