We've heard - and experienced - a lot of discussion about the technology hurdles of remote testing: picking the best testing tool, getting participants to download screensharing tools, dealing with proprietary software that participants don't have access to...Tech issues run the gamut.
But we hear considerably less about the interpersonal challenges of running a remote usability session. It's not just about finding the right tool or getting your participant to download Webex; it's about maintaining your participant's motivation and engagement by showing (gasp!) empathy and compassion during a remote session.
Why, you may ask, are we inclined not to show empathy and compassion while moderating? Why do we adopt that passive, less-than-friendly approach for in-person tests? Well, that's tried-and-true: Participants sitting next to you shouldn't be your friend; if they're your friend, they're more likely to try to please you, and that means biased responses and biased feedback. You should be so neutral and mild that it almost seems like you're not even in the room with the participant. They should feel like they're just hanging out on the Internet, completing some tasks, talking to themselves. That's the moderator's role in the traditional usability test.
But what happens when your participant is across the state, the country, or the world?
"Hang on - let me take this call real quick."
"Hang on - let me pop open the system we currently use so I can show you how awful it is."
"Hang on - actually, can we cut this short? A tee time just opened up at my golf course."
This, of course, is very different from our experiences with in-person usability testing, where the participant is essentially trapped in a room with us and we're calling the shots. With face-to-face sessions, we've got nearly undivided attention. There's no personal email, no bosses demanding attention, no bookmarked websites that users want to complain about.
So why the drastic contrast? Well, we've done some experimenting, and we've confirmed some theories.
What if it's us, the moderators?
Maybe it's not the tech hurdles or the distractions or the time commitment. Maybe - just maybe - that super-passive persona we assume as moderators of an in-person test (neutral, reserved, non-emotional) is not effective at maintaining users' attention or willingness to complete a study remotely. Maybe because the participant and moderator are interacting via a computer, remote users treat a neutral moderator like...a computer—and not in an endearing C3PO or Rosie Jetson way.
Participant: "Oh, I didn't see that button at first...but I'm old...so that's probably why."
Moderator: "You're not old, Barbara; you're just experienced! Let's try the next task."
Moderator: "What do you expect to happen after you select 'Save'?"
Participant: "Oh! Sorry; I already selected it."
Moderator: "You're too fast for me! Can you cover your eyes?"
Participant: "Hahaha. I will just press the back button."
So, at the end of the day, even if the tasks, protocol, and recruitment criteria are the same for a group of in-person participants and a group of remote participants, the rapport you build with participants online needs to be much more personable and friendly in order to get the type of high-value feedback that emerges from in-person tests. Showing your remote participants a little bit of empathy works wonders to keep them on track and get them talking - and they're less likely to try to please you because you're still a world away from them.