Determining if difference is significant I work on an ecommerce site, and I'm trying to determine a feature's impact on conversion.
This is not an AB test; rather, 100% of customers had access to the feature- whether or not they used the feature is indicated in "Segment" below.
In the past 12 months:




Segment
Converted
Didn't convert
Total




Used feature
629
20,570
21,097


Didn't use feature
243,273
8,131,398
8,374,671




Plain English: Of 21,097 customers who used feature, 629 or 2.98% converted. Of 8,374,671 who didn't use feature, 243,273 or 2.90% converted.
At first glance "Used feature" has higher conversion, but I'm wondering what other calculations I need to make a statistically sound conclusion.
New to stats.stackexchange so apologies if formatting is improper!

Normally, testing involving two samples requires more complicated analysis than testing with one sample. But in this case, the sizes are so unbalanced that it's a reasonable approximation to just do an analysis on the smaller sample. We can take a null hypothesis that the smaller sample is from a Poisson distribution whose $\lambda$ is equal to the percentage of the large sample times the size of the small sample. That gives $0.029*20570=596.53$. This gives a p-value of 8.9%, which is not significant. You could also use a binomial distribution, which gives a p-value of 8.6%.
