Virtual approaches to device testing | Branding Science
Posted in December 2021
Dr. Anne-Sophie Lenoir
Director DACH, Branding Science
Eighteen months into the pandemic, in-person research continues to be significantly curtailed in many markets. While many types of projects could be relatively easily adapted to a new remote environment, this has proven especially challenging for hands-on, observational research including prototype and usability testing. How can we continue to ensure we fully capture user needs, workflows, and experiences, and effectively support clients in making design decisions with confidence in a new reality?
Traditionally, in-person ethnographic research has focused on observing healthcare providers and patients as they use the device, asking questions as needed during the procedure. This typically takes place in medical settings or in the patient’s home, which also allows for deep insights into the use environment. In contrast, in-person interviews in central locations tend to be more structured in nature, but still enable the research team to directly observe the participant and their body language as they manipulate the device.
Three virtual observation approaches offer remote alternatives to these techniques:
WATI device testing
This approach leverages a standard WATI (web-assisted telephone interview) platform, including video via webcam, to observe the respondent as they manipulate the product during a structured interview. Devices and disposables are shipped to the respondent, who sends them back to the research team after testing. This format enables the moderator to interact with the respondent, providing instructions and probing where appropriate; depending on local compliance requirements, clients may also be able to watch the interview. Limitations include a set timeframe, relatively low image quality, a single point of view, and the risk of encountering technical issues.
When to use?
In lieu of IDIs (structured questions and/or complex instructions), with added geographic flexibility; webcamquality video / fixed point of view is sufficient
Mobile ethnography
Mobile ethnography relies on respondents recording videos of themselves and their environment as they use the device, taking pictures, and optionally completing other tasks via an app on their smartphone. The moderator can probe on specific topics after the content has been posted. Engagements typically take place over the course of a week or more, with multiple respondents from a given market using the platform during the period. This is often complemented with follow-up interviews.
This approach enables longer observation periods as well as multiple observations over time in an authentic use environment and empowers the respondents to take part at a time convenient for them. Potential challenges include the requirement for respondents to own a smartphone and be comfortable using digital tools, which can be problematic in some populations, as well the need to maintain respondent engagement over a longer period. A skilled fieldwork team experienced with this type of research is essential.
When to use?
Exploratory research – Need for a broader understanding of the context, greater depth, and/or multiple data points through an extended engagement
Virtual observation
Virtual observation involves shipping a recording device or camera to the respondent, to be used either with a head strap of with a stand. Once again, respondents record themselves at their convenience and in their own environment, but the point of view more closely stimulates the user’s own perspective or that of in-person observation. This approach enables higher-quality video recordings as well as multiple points of view; however, there is no interaction with the respondent during the research, so clear instructions and pilots are paramount.
When to use?
Need for high-quality video materials and/or a specific point of view; long procedures where moderation is not required
Conclusions
Virtual observation techniques enable researchers and device manufacturers to gain deep insights into users’ needs and experiences, in their own environment and regardless of geographic constraints. Because of this, we expect that these methods will continue to play an important role in the longer term, even as in-person research progressively resumes.
However, some challenges remain. These methods all involve shipping devices to the respondent via courier; this is convenient and cost-effective for relatively small devices but may not be appropriate when large or high-value devices are being tested.
We should also keep in mind that these approaches require respondents to be at least somewhat comfortable with technology. This may prove a challenge particularly in the context of patient research. Focusing solely on digitally savvy participants can result in missed user needs and biased recommendations. To avoid this, researchers should strive for inclusive research design and recruitment and consider involving caregivers where appropriate.
Download
For more information or inquiries about our work in the MedTech and Diagnostics spaces, please contact Anne-Sophie Lenoir (anne-sophie.lenoir@branding-science.com)
