From A/B testing to Learning Launch

Challenge

As the main software engineer of a small startup with limited time and resources, my biggest  challenge is to create and improve the right features of our product, a social app, in order to increase  the interaction between the users and the recommendation of the app to friends, in other words,  release what users will love and engage with. The product development team is very small, just  three developers, as well as one designer, so there is no enough resources to conduct long user  researches. We need to be fast and assertive with our releases.

Selection

We are already experimenting some practices to speed up our releases and improve the results  (engagement and sharing), the last one was A/B testing, but with some trade-offs, like a long time to  prepare multiple tests and to verify multiple results . After some time considering many agile  methods to our prototyping tests and knowing Design Think, the learning launch tool makes sense  to us because it is an experiment meant to quickly test an idea and prototype, focusing in one idea,  not two or more.

Application

We have a lot of ideas to improve our metrics in our product backlog. These ideas are ordered in  accordance to some assumptions, like users reviews and some observations. Now, before to create  multiple features to A/B testing,  we take one idea from our product backlog and create a fast mock  to show that feature to a vast number of users to get their reviews. If they show us enthusiasm, we  go to the next level and we create a new release of the product with that feature. With A/B testing  we spend a couple of days to test two ideas, now we can test many ideas in a couple of days and the  result is very assertive, we improve our metrics with less resources.

Insight

The most important thing to us by applying the learning launch tool and focusing in one idea was  that we removed the A/B testing's comparator bias and our mindset was guided to think about that  idea in a broader way. When everyone in the team is concentrated in one idea, we can have a better  synergy and collaboration, we really work as a team and we bring everyone to the problem. The  focus is one idea, not a war to between ideas. The result is the improvement of the metrics that  really matter, the user engagement and sharing.

Approach

We switch from A/B testing to the launch and learning tool because this learning launch tool  perfectly meets our needs in relation to the time and results it provides, besides that, we gained a lot  of knowledge and insights when we have all members of the team focused in one idea. We have  used this tool for some iterations and haven't made any modiﬁcations to the workﬂow since.

