Challenge

My company is a large B2B ecommerce website. While we sell to other businesses, we have  B2C sensibilities: meaning we sell mostly to purchasing/office/safety managers, these are not  contracted sales, and the buyers can easily order from anyone else in the industry.

Free shipping is a very popular teaser for ecommerce.  You can find options from free shipping  on any order size, free on $99, free on $299, free on $999 and even higher.   Many people may  not know this, but markup on shipping rates is (or can be) a very lucrative revenue stream for  ecommerce shops, especially in B2B.  And if you ship lots of packages, like we do (1,000 per  day).  Markup of $4 over cost of shipping on all orders and you just made an extra $1,000,000  (yearly) in profit direct to the bottom line.

So naturally with this amount of revenue at risk, we wanted to carefully explore the free  shipping threshold we will offer to our customers, exceedingly expecting to see free shipping at  some price point.

Selection

We decided to use the Learning Launch tool.  Since we realize this will certainly be a very  quantitative analysis, making this a strict, randomized test makes the most sense.

Furthermore, with proper segmentation and measurement, we will be able to answer the most  key question we need answered:

• not which free shipping threshold creates the highest conversion rate or number of  orders in a given day,  • but which free shipping threshold maximizes the total revenue for the company  inclusive of the shipping revenue.

With this approach being an experiment, we can structure it as an A/B(C/D/E) test.

Application

In running this as an experiment, we randomly segmented our daily visitors into the following  experiences:

• Free shipping on any order (no minimum)  • Free shipping on $49  • Free shipping on $99  • Free shipping on $199  • No free shipping at all

Decisions on the random assignment of visitors happened in real-time (assuming the user has  not already been assigned) before the first page loads.  Site messaging is all customized and  speaks to the particular offer/segment the visitor is in.  The segment assignment is persistent:  users who order or leave without ordering, will see the same segmented offer when they  return, for the duration of the test.

Insight

Again, since this was an experiment, we were able to determine at a statistically significant level  (95% confidence) which option maximized average order value.

We ran this experiment for about six weeks to determine a winner at the generally accepted  confidence level of 95%.  The results of the experiment told us that in order to maximize  average order value, we needed to only offer free shipping to shopping carts of $199 or more.   On the surface, it would seem that we’re better off charging shipping on most orders (as most  are under $199, sitewide AOV ~ $167), and only offering free shipping when the order exceeds  our sitewide average.  This is how we declared our winner and is in fact partially true.

But, it was after further examination on the profitability of each order that we confirmed our  findings.  Remember, markup on shipping is pure profit.  Even though our products have decent  profit margins, more orders with a shipping cost raised overall net profit.

Approach

Caution: these findings might not be appropriate for other retailers.  For one thing, you might  be willing to process orders very extremely slim margins (gasp- even a loss!) in order to grow  your customer base.  This was Amazon’s strategy early on.

For us as a mature etailer and healthy customer base, maximizing revenue was what we were  exploring.  And I’ll say it again, maximizing revenue may not be your immediate goal.  Customer  satisfaction, active customers, repeat buyers and so many more metrics may be your  immediate goal.

In considering the others tools we learned of this course, no other could have suited us and the  question we were trying to answer.

