One of the ways of learning a new language is by going to another place and immersing yourself in that environment. Ogma is a Virtual Reality System in which you can immerse yourself without going anywhere else and learn a new language.
Project Inspiration: In Fall 2015, my lab purchased the new Oculus DK2 and I wanted to use it for my Masters Thesis project. So I, along with an undergraduate researcher, decided to use VR in the context of language learning. That's how Ogma became a reality!
Suppose you want to learn german. You decide to go and live in Germany for few months so that you can immerse yourself in the environment and not only learn german but also the culture. This is call 'Immersion Method of Learning a New Language'.
Immersion is proven to be one of the most effective ways to learn language. However, it isn’t always a feasible option, so less effective traditional learning methods are used. But these modern language learning methods are still dramatically worse than direct immersion.
IDEA: Virtual reality could simulate this immersion to provide a more natural and effective language learning method.
"Could virtual reality be used to improve retention of new vocabulary via immersion?"
Virtual Reality is fully immersive computer simulated environment that gives user the feeling of actually being there. We were trying build an immersive virtual environment for language learning. With this, we wanted to introduce a new way to learn a language and test the feasibility of VR environments as training platforms.
There are various disadvantages of immersion method - Time, Money, Danger (Learning Syrian), Weather Conditions, Job!
We conducted literature review to find if similar research study have been done in the past. Virtual Reality was used as learning environment but they were all cave systems. We did not find any work using Head Mounted Display.
To prove this thesis, we set on a journey using the Iterative User Centered Design Process.
I followed the ITERATIVE User Centered Design Process for this project.
There were 4 Iterations, each consisting of Research, Design and Testing Phase and a Final User Study.
I interviewed 7 people, 4 student taking a language class, 2 working people who had been to other country for work and 1 person who had enrolled in a french class just as a hobby.
How fast do they learn?
How existing language learning system work?
How VR can improve language learning experience?
Who are the audience/user/stakeholders of the system?
For better understanding of the users, I created personas.
Based on the answers, brainstorming was done for initial system design. An INTERACTIVE VR environment in which the user can move around, interact with the world and learn new language words was planned.
Aim: To find out how many words would be appropriate for the final system.
Approach: We did our initial pilot study on 3 participants.
Participant A was given this list and was asked to learn them in 5 minutes just by looking at the list. Then the participant used a vocabulary training website (vokabel.com) to practice. Participant A was tested on the words immediately after as well as a week later, and scored 9/10 and 5/10, respectively.
Participant B and C were also given the same list and were asked to learn them in 5 minutes by looking at the list. Participant B was tested immediately after and scored 4/10. Participant C was tested the next day and scored 8/10.
Observations: Participants had more difficulty with words that were different from English. For example, nobody got water (vatten) correct in the test. In addition, participants tend to remember more if they are given some kind of practice. Thus, we decided to use games in our system to help users learn these words.
What we learned: Considering the lack of training presented in this pilot study, we have decided that 10 words should be a reasonably sized lexicon to use in our study. We also learned that time plays a major role. There is a difference between holding words in memory and actually learning them. Therefore, it would be ideal for us to provide our assessment at least a week after the training.
Low fidelity prototype was made to check the validity of our idea. A real room was simulated as our virtual environment. Objects had been labelled with Swedish text.
Wizard-of-oz technique was used for this user study. 3 participants were asked to explore the room. A soft music was on in the background. As the participants explored the room, they saw textual cues (other language) over some object. When they looked at it, the music stopped playing and they heard an audio cue (other language) saying the object’s name. After this exercise the participants were tested.
All of the participants remembered the object’s name in the other language very well. They liked the music as well. One of them asked "How will you make a person walk in VR?"
The question asked by one of the participants made me think about interaction. Interaction was difficult (Remember, this was late 2015 and VR was very new. The only easy way to interact with VR environment was by using the keyboard, which was not at all user friendly). We explored two options - Leap Motion and Myo Band.
Myo band was decided to be used over leap motion as it was a better input for VR environment. Medium fidelity prototypes, also known as digital prototype was built to find the user’s reaction to VR environment. An open VR environment was created.
User Study: I conducted an informal user study with 3 participants. The users were asked to move in VE by wearing Myo band. They were asked to look at a cube and read the word displayed. We tested our hardware (Oculus and Myo) in this environment.
User Feedback: The users felt uncomfortable navigating in the VE wearing Myo band they complained that the gestures used for navigating were difficult to maintain in VE. This was an attempt to find the easy of interaction with VE.
I created the 3D environment. It was an apartment having living room, bedroom, kitchen and balcony. The space was created so that the user can move around and interact with the objects. 10 objects were placed in the apartment. Myo interaction was also built.
“Music is good, but should be instrument - no lyrics” The user liked the concept of music but suggested it should be instrumental as it will not interfere with the pronunciation of the words.
“Oculus is too heavy”. The DK2 headset was heavy and wired. This caused discomfort.
"Need for practice environment”. The user felt very uncomfortable directly dealing with the new VE without any practice. A need for practice environment became necessary.
“Slower pronunciation” The pronunciation were fast for some users.
The following changes were done based on user feedbacks.
Instrumental Music was added as in the learning environment.
Easy to recognize without English translation – 10 Swedish objects were chosen.
Final User Testing was done before Data collection. One participant tested the system. After using the system, a small assessment was done to check how many words he remembered.Below tables shows the performance of that participant.
Phase 1: Learn how to interact with VR environment using Myo Band and Oculus rift in the practice environment.
Phase 2: Freely roam a virtual apartment space. When looking at a relevant object, text will be shown of what it is in the Swedish language along with audio cues.
Phase 3: Game session in which user is given the word and they must go find the object.
Final VR System was made as per user feedback. The interaction using Myo band was adjusted as well to keep the system user friendly.
I designed the user study (A/B Testing) in the following manner.
Then immediately after learning the words, the participants will give a test.
After one week, the participants will be called and asked to give the test again.
After completing the test, they will fill out the questionnaire & give informal feedback.
36 student volunteers were chosen as participants out of which nineteen participants successfully completed the user study. They were randomly placed in either the traditional or virtual reality group.
Traditional group participants were first given a list of ten Swedish words and their English equivalents and told to study them for up to five minutes. The words were read out loud to them, repeated upon request. Then, they were given flash cards corresponding to the words, and told to study them until they felt they were ready for the test.
Virtual reality group participants entered a virtual practice environment where they could become accustomed to navigating virtual reality. Then, they entered the virtual apartment and were allowed to explore freely for up to five minutes, looking at objects to see and hear the corresponding Swedish words. Afterward, they entered the second phase, in which the Swedish words were displayed and pronounced and participants had to point to the corresponding object, until each was complete.
Test: Both groups took a test in which they were given the list of English words and told to write and pronounce the corresponding Swedish words. For each word, participants were graded on both spelling and pronunciation with correct(5 points), almost correct(2.5 points), and incorrect(0 point). All participants came after a week, gave the same test and filled out the questionnaire.
Have they learned any language before?
Which other methods did they use for learning language?
How was it different from our method?
We measured the scores from the initial test (the test given immediately after the study) & post test (the test given one week later). We also calculated the word memory retention.
Ratings were much higher for virtual reality than for the traditional method, with the majority of participants rating virtual reality as “Very Enjoyable” & "Very Effective".
Even though traditional method had higher initial scores, the retention of VR system was way higher than traditional. VR is better than traditional because “it allows them to visualize objects”.
The findings show excellent potential for virtual reality in language acquisition.
I published this work in an international academic conference - PETRA in 2016. You can find the paper here.
I was awarded the 'Verizon Outstanding Masters Thesis Award' in 2016 for this project. Ogma became quite popular in my university as it was the first Virtual Reality System developed using Iterative User Centered Design Process.