Re-Engaging  Students:  A  Learning  Launch  Experiment

Salman Fong,  June  27  2020

1. Challenge

At   an   EdTech   company   that   I   worked   at,   we   were   undergoing   a   massive   digital   transformation   program.   Our   team   was   tasked   with   finding   innovative   solutions   that   we   can   bring   to   the   market   quickly   to   broaden   our   offering   by   utilizing   our   existing   content.   The   company   was   a   traditional   publishing   house   catering   textbooks   to   a   large   number   of   schools   &   universities.   It   was   not   easy   to   change   the   existing   business   processes.   The   backend   content   creation,   organization   and   data-management   was   a   time-consuming   process.   But   it  provided   the   high   quality   content   that   the   market   had   come   to   expect   of   the   company.   We   cannot   compromise   on   the   quality   for   sure,   but   we   need   to   streamline   the   process   to   be   a   bit   more   nimble   and   fast.   The   turnaround   time   for   material   revision   was   in   the   order   of   months   if  not   years,   which   meant   that   we   cannot   deliver   up-to-date   content   to   the   students   in   a   timely   manner.   Another   more   important   challenge   that   was   facing   the   academic   institutions   was   the   high   number   of   dropout   rates.   Students   just   stopped   attending   schools.   Reasons   varied   -   lack   of   interest/motivation,   socio-economic   factors   etc.   And   that’s  the   challenge   that   we   were   willing   to   take   on   initially.   How   can   we   keep   the   students   engaged   and   motivated   so   they   don’t   drop   out   -   since   this   has   a   huge   impact   on   their   future   career   and   on   the   community   as   a   whole.            Even   though   this   concern   had   been   in   the   minds   of   the   business   executives   for   a   while   and   there   were   various   ideation   exercises   taken   up,   none   of   them   were   put   to   practice,   as   they   were   stuck   in   the   processes.   The   legacy   teams   were   not   adopting   the   approaches   from  the   Design   Thinking   framework.   When   I   joined   the   company   it  was   in   the   mid-2010s.   My   small   team   was   excited   about   applying   newer   methodologies   to   these   burning   issues.   We   noticed   the   increasing   significance   mobile   phones   were   playing   in   our   lives   in   general,   and   students’   lives   in   particular.   Our   team   started   observing   the   effect   of   smartphones   on   students,   and   how   the   smartphones   had   become   so   ubiquitous   and   so   personal.   Majority   of   the   students   relied   on   these   devices   as   primary   source   of   information/entertainment/distractions!   So   we   decided   to   develop   a   prototype   mobile   app   that   can   deliver   key   information   to   them   at   the   right   times   so   they   can   stay   on   top   of   their   course   assignments.   This   was   to   be   our   ‘learning   launch’   experiment!   We   had   come   up   with   a   few   hypotheses   to   test:   (1)   Friendly   reminders,   delivered   via   smartphones,   to   students   regarding   their   upcoming   assignments   is   going   to   help   them   stay   on   track,    (2)   We   could   gain   more   insight   into   the   users’   app   engagement   which   can   further   drive   feature   developments,   (3)   We   could   make   future   content   delivery   ‘mobile-first’   for   an   optimal   UX   and   improved   engagement.

2. Selection

Based   on   the   challenges   our   team   faced,   and   the   overall   course   of   our   business

transformation,   we   decided   to   use   the   ‘learning   launch’   design   thinking   tool   for   our   purposes.   As   stated    here ,   learning   launch   is   a   quick   and   inexpensive   experiment   to   validate   a   growth   idea   before   committing   additional   resources   to   build   out   a   full   product.   We   chose   this   tool   because   of   it’s   real-world   driven   feedback   that   can   help   us   iteratively   improve   upon   the   initial   ideas.   It   can   also   help   us   park   our   disproven   hypotheses   for   the   time   being   and   ensure   we   focus   on   the   really   valuable   ones   that   meet   the   current   market   conditions.        Following   Michael Just’s   instructions    as   stated   in   Module   4/Week   4   of   the   Design   Thinking   for   Innovation   course,   we   focussed   on   testing   out   the   various   hypotheses   to   get   to   the   different   ‘trues’.   We   had   to   pick   the   best   idea(s)   to   work   on   to   have   a   positive   business   impact.   We   wanted   to   avoid   a   lot   of   guesswork   and   incorrect   projections   just   so   we   can   come   up   with   a   business   plan.   We   wanted   to   learn   quickly   from  our   ideas.   We   wanted   to   identify   the   ones   that   showcase   the   ‘burning   need’   of   our   customers.   We   wanted   to   know   the   real-world   truths   about   the   various   ideas   we   were   exploring.   These   reasons   helped   us   zero   in   on   the   learning   launch   tool   as   the   preferred   one   for   this   experiment.

3. Application

In  order  to  apply  this  model,  we  committed  to  building  a  prototype  app  first.  It  was  to  be

responsive  with  less  barrier  for  adoption  by  students.  With  minimal  resources  -  small  team,   shortened  time  period,  AND  not  much  of  a  budget  -  we  went  about  building  a  responsive  mobile   app  that  was  bare  minimal  in  functionality.  From  a  broad  set  of  features  we  had  originally   planned  for  a  full  fledged  mobile  app,  we  pared  down  to  the  basics.  We  ended  up  deciding  to   support  just  three  key  features:  (1)  Login  for  enrolled  users  only,  (2)  View  course  info,  along  with   assignment  dates  (3)  Opt-in  for  reminders  and  notifications.   This  helped  us  to  clearly  get  the   relevant  usage  data  to  prove/dis-prove  our  hypotheses  that  we  set  out  to  test.     Around  100  pre-approved  users  were  given  access  to  the  app.  This  helped  us  keep  the  user   count  to  a  manageable  level.  We  were  able  to  integrate  certain  analytics  features  into  the  app   so  we  can  collect  meaningful  data  for  our  analyses.  That’s  where  our  learning  would  kick  in.  As   an  alternative  mode  of  data  collection,  we  were  able  to  bring  in  a  few  student  volunteers  to   observe  first  hand  how  they  used  our  app,  with  candid  feedback.      With  both  the  modes  we  were  able  to  monitor  &  gather  a  few  weeks  worth  of  data,  which   revealed  to  us  the  various  usage  patterns,  shortcomings  and  the  missing  features.     We  really  wanted  to  test  our  assumptions  about  the  friendly  reminders  helping  them,  and   whether  the  students  prefer  nuggets  of  content  optimized  for  mobile  model  of  interaction  -  brief   short  duration  vs  long  duration  assignments.

4. Insight

Throughout  this  rollout  our  team  was  committed  to  providing  the  best  possible  experience   and  to  gather  the  maximum  quality  data  for  future  analysis.  Applying  learning  launch  helped  us   bring  a  certain  rigor  and  discipline  into  prototype  building  &  data  collection  processes.  It   certainly  helped  me  see  the  value  of  design  thinking  as  a  key  framework  in  the  business   context.        As  we  started  interpreting  the  collected  data,  we  observed  key  patterns  that  validated  our   hypotheses!  Users  certainly  liked  the  reminders  -  almost  all  of  them  opted-in  right  away  -  that   helped  them  complete  and  turn-in  their  assignments  on  time.  They  really  were  spending  more   time  in  the  reminders  screen  of  the  app  than  any  other.  Another  interesting  outcome  was  that   they  really  preferred  short  assignments  to  long  ones.  Given  the  nature  of  frequent  distractions   when  using  a  smartphone  it  was  no  surprise  that  short  bursts  of  key  engagements  provide  a   better  experience  for  the  students.  That  in  turn  keeps  them  motivated  to  learn  a  few  concepts  at   a  time.  This  turned  out  to  be  a  very  useful  insight  in  the  whole  experiment.  Till  this  point  our   content  creators  were  focussed  on  creating  a  huge  blob  of  linearly  delivered  course  material.   But  this  experiment  and  its  results  highlighted  the  changing  learning  landscape.  This  meant  that   our  business  has  to  quickly  adapt  to  the  market  conditions  and  produce  content  to  meet  those   conditions.  We  did  have  the  content,  we  just  need  to  ‘bite-size’  the  content  so  we  can  keep  the   students  coming  back  for  more  thereby  helping  them  achieve  their  learning  goals!     That   last   value   alone   proved(to   me,   atleast)   how   worthwhile   design   thinking   is!  In   a   traditional   world   people   would   have   hypothesized   this   model,   but   would   never   have   ventured   to   test   it  out   in   the   real   world   at   such   a   micro-scale.   To   me   this   was   an   eye-opener.   Since   this   experiment,   I’ve   been   keen   to   apply   the   various   tools   of   Design   Thinking   to   optimize   the   execution   time   of   an   idea   -   ideate,   prototype   &   validate,   iterate-repeat,   release.   Certainly   learning   launch   is   an   invaluable   tool   in   the   arsenal!

5. Approach

Before  we  started  down  this  path,  our  go-to  design  thinking  tool  was  the  mind  mapping  tool.  It   is  valuable  in  its  own  right  in  order  to  get  an  understanding  of  ‘What  is?’   But  for  our  challenge   here,  we  opted  for  the  learning  launch  tool  since  we  had  a  clear  objective  to  apply  it  to.  We  had   a  few  assumptions  to  test  out,  and  we  were  at  a  key  point  in  the  product  life  cycle  where  we  had   to  prove  if  the  new  highly-impactful  idea  would  work  in  the  real  world.      If  I  were  to  do  this  again,  I’d  definitely  pick  the  learning  launch  tool,  as  it  clearly  satisfied  our   needs.  It  provided  us  the  data  we  needed  to  convince  the  business  stakeholders  to  change   course  and  support  our  team  with  more  resources  for  real  product  build  out.  That  validation  was

crucial.  It  also  helped  our  team  focus  on  the  right  set  of  data  to  use  for  analysis,  and  also   prodded  us  to  ask  the  right  questions  in  order  to  get  the  best  insights.  Plus,  everyone  in  the   team  enjoyed  the  discovery  &  the  learning  process.  It  kept  us  engaged  and  excited!     And,  at  the  end  of  this  experiment  we  set  out  to  build  the  Release  1  of  our  product,  improving   upon  the  prototype!  This  time  with  a  clear  roadmap  of  new  features  &  milestones  per  the  market   research.           Salman Fong   June  27,  2020


Re - Engaging Students : A Learning Launch Experiment Salman Fong , June 27 2020 1 . Challenge At an EdTech company that I worked at , we were undergoing a massive digital transformation program . Our team was tasked with finding innovative solutions that we can bring to the market quickly to broaden our offering by utilizing our existing content . The company was a traditional publishing house catering textbooks to a large number of schools & universities . It was not easy to change the existing business processes . The backend content creation , organization and data - management was a time - consuming process . But it provided the high quality content that the market had come to expect of the company . We can not compromise on the quality for sure , but we need to streamline the process to be a bit more nimble and fast . The turnaround time for material revision was in the order of months if not years , which meant that we can not deliver up - to - date content to the students in a timely manner . Another more important challenge that was facing the academic institutions was the high number of dropout rates . Students just stopped attending schools . Reasons varied - lack of interest / motivation , socio - economic factors etc . And that ’s the challenge that we were willing to take on initially . How can we keep the students engaged and motivated so they do n’t drop out - since this has a huge impact on their future career and on the community as a whole . Even though this concern had been in the minds of the business executives for a while and there were various ideation exercises taken up , none of them were put to practice , as they were stuck in the processes . The legacy teams were not adopting the approaches from the Design Thinking framework . When I joined the company it was in the mid-2010s . My small team was excited about applying newer methodologies to these burning issues . We noticed the increasing significance mobile phones were playing in our lives in general , and students ’ lives in particular . Our team started observing the effect of smartphones on students , and how the smartphones had become so ubiquitous and so personal . Majority of the students relied on these devices as primary source of information / entertainment / distractions ! So we decided to develop a prototype mobile app that can deliver key information to them at the right times so they can stay on top of their course assignments . This was to be our ‘ learning launch ’ experiment ! We had come up with a few hypotheses to test : ( 1 ) Friendly reminders , delivered via smartphones , to students regarding their upcoming assignments is going to help them stay on track , ( 2 ) We could gain more insight into the users ’ app engagement which can further drive feature developments , ( 3 ) We could make future content delivery ‘ mobile - first ’ for an optimal UX and improved engagement . 2 . Selection Based on the challenges our team faced , and the overall course of our business transformation , we decided to use the ‘ learning launch ’ design thinking tool for our purposes . As stated here , learning launch is a quick and inexpensive experiment to validate a growth idea before committing additional resources to build out a full product . We chose this tool because of it ’s real - world driven feedback that can help us iteratively improve upon the initial ideas . It can also help us park our disproven hypotheses for the time being and ensure we focus on the really valuable ones that meet the current market conditions . Following Michael Just ’s instructions as stated in Module 4 / Week 4 of the Design Thinking for Innovation course , we focussed on testing out the various hypotheses to get to the different ‘ trues ’ . We had to pick the best idea(s ) to work on to have a positive business impact . We wanted to avoid a lot of guesswork and incorrect projections just so we can come up with a business plan . We wanted to learn quickly from our ideas . We wanted to identify the ones that showcase the ‘ burning need ’ of our customers . We wanted to know the real - world truths about the various ideas we were exploring . These reasons helped us zero in on the learning launch tool as the preferred one for this experiment . 3 . Application In order to apply this model , we committed to building a prototype app first . It was to be responsive with less barrier for adoption by students . With minimal resources - small team , shortened time period , AND not much of a budget - we went about building a responsive mobile app that was bare minimal in functionality . From a broad set of features we had originally planned for a full fledged mobile app , we pared down to the basics . We ended up deciding to support just three key features : ( 1 ) Login for enrolled users only , ( 2 ) View course info , along with assignment dates ( 3 ) Opt - in for reminders and notifications . This helped us to clearly get the relevant usage data to prove / dis - prove our hypotheses that we set out to test . Around 100 pre - approved users were given access to the app . This helped us keep the user count to a manageable level . We were able to integrate certain analytics features into the app so we can collect meaningful data for our analyses . That ’s where our learning would kick in . As an alternative mode of data collection , we were able to bring in a few student volunteers to observe first hand how they used our app , with candid feedback . With both the modes we were able to monitor & gather a few weeks worth of data , which revealed to us the various usage patterns , shortcomings and the missing features . We really wanted to test our assumptions about the friendly reminders helping them , and whether the students prefer nuggets of content optimized for mobile model of interaction - brief short duration vs long duration assignments . 4 . Insight Throughout this rollout our team was committed to providing the best possible experience and to gather the maximum quality data for future analysis . Applying learning launch helped us bring a certain rigor and discipline into prototype building & data collection processes . It certainly helped me see the value of design thinking as a key framework in the business context . As we started interpreting the collected data , we observed key patterns that validated our hypotheses ! Users certainly liked the reminders - almost all of them opted - in right away - that helped them complete and turn - in their assignments on time . They really were spending more time in the reminders screen of the app than any other . Another interesting outcome was that they really preferred short assignments to long ones . Given the nature of frequent distractions when using a smartphone it was no surprise that short bursts of key engagements provide a better experience for the students . That in turn keeps them motivated to learn a few concepts at a time . This turned out to be a very useful insight in the whole experiment . Till this point our content creators were focussed on creating a huge blob of linearly delivered course material . But this experiment and its results highlighted the changing learning landscape . This meant that our business has to quickly adapt to the market conditions and produce content to meet those conditions . We did have the content , we just need to ‘ bite - size ’ the content so we can keep the students coming back for more thereby helping them achieve their learning goals ! That last value alone proved(to me , atleast ) how worthwhile design thinking is ! In a traditional world people would have hypothesized this model , but would never have ventured to test it out in the real world at such a micro - scale . To me this was an eye - opener . Since this experiment , I ’ve been keen to apply the various tools of Design Thinking to optimize the execution time of an idea - ideate , prototype & validate , iterate - repeat , release . Certainly learning launch is an invaluable tool in the arsenal ! 5 . Approach Before we started down this path , our go - to design thinking tool was the mind mapping tool . It is valuable in its own right in order to get an understanding of ‘ What is ? ’ But for our challenge here , we opted for the learning launch tool since we had a clear objective to apply it to . We had a few assumptions to test out , and we were at a key point in the product life cycle where we had to prove if the new highly - impactful idea would work in the real world . If I were to do this again , I ’d definitely pick the learning launch tool , as it clearly satisfied our needs . It provided us the data we needed to convince the business stakeholders to change course and support our team with more resources for real product build out . That validation was crucial . It also helped our team focus on the right set of data to use for analysis , and also prodded us to ask the right questions in order to get the best insights . Plus , everyone in the team enjoyed the discovery & the learning process . It kept us engaged and excited ! And , at the end of this experiment we set out to build the Release 1 of our product , improving upon the prototype ! This time with a clear roadmap of new features & milestones per the market research . Salman Fong June 27 , 2020