Determining if difference is significant I work on an ecommerce site, and I'm trying to determine a feature's impact on conversion.
This is not an AB test; rather, 100% of customers had access to the feature- whether or not they used the feature is indicated in "Segment" below.
In the past 12 months:




Segment
Converted
Didn't convert
Total




Used feature
629
20,570
21,097


Didn't use feature
243,273
8,131,398
8,374,671




Plain English: Of 21,097 customers who used feature, 629 or 2.98% converted. Of 8,374,671 who didn't use feature, 243,273 or 2.90% converted.
At first glance "Used feature" has higher conversion, but I'm wondering what other calculations I need to make a statistically sound conclusion.
New to stats.stackexchange so apologies if formatting is improper!

If all the customers are independent (e.g. you don't have data on the same customer multiple encounters) then this is the classic 2 by 2 table that can be analyzed using a Chi-Squared test: https://en.wikipedia.org/wiki/Chi-squared_test.
