What works in e-commerce – A meta-analysis of online experiments [pdf]
Very interesting thanks for sharing. As someone working in e-commerce, I was smiling when I saw some of them but it is frustrating on how often these ineffective experiments are repeated.I would have loved to see the cut of performance by industry/sector. My hunch is some of the things would work really well in travel but not as well in others especially low involvement categories and categories with lower average selling price. It would also be interesting to know the average duration of these A/B tests, I think some of things like scarcity and urgency will have larger effect in the shorter time duration vs others like UI changes which will take a while to produce substantial results, mostly because customers will have to learn new behaviours. Product recommendations is interesting because it is notoriously difficult to get them right and feel they tend to work better in long tail categories like media vs. head heavy categories like mobiles or laptops. They may also not work well in categories where brand influence is high and are generally high involvement and high cost.