An Exploration on the Inclusion of Mobile Audiences and Its Impact on Insights
For several months, I’ve been part of an effort spearheaded by GRBN to address the participant experience – to ensure that not only is it a pleasant and inviting experience, but that more people are enticed to participate.
One of the areas that stands out the most when you look at the experience of survey participants is mobile. Surveys that look like this:
And the litany of other headaches that go with trying to complete surveys designed for desktop users on a mobile device.
To address this, I’ve been part of a team of market research professionals conducting research-on-research (ROR) that brings together clients, agencies, data collectors and technology providers across the globe, with the intent of creating meaningful change.
We released a handbook, ENGAGE – 101 tips to improve the research participant user experience – alongside specific metrics to watch, and the ROR continues with the intent of continuously improving the experience.
Looking at the Mobile Experience
Taking what we developed for ENGAGE in the ROR process and applying it to the mobile experience, I was part of a panel recently in which we explored inclusion of mobile audiences and its impact on insights.
This evaluation of a possible course correction for conjoint analysis was part of the Insights Association NEXT Conference in May (watch the webinar playback from a recent encore presentation here). It allowed us to delve into the specific needs of a company, Mindbody, that wanted to better understand dynamic pricing for their mobile audience, similar to what Uber offers.
Mindbody had three questions it needed to ask:
- The impact of price on behavior
- The variables that influenced participants’ willingness to sign-up for a yoga class (time of day, day of week, type of yoga, instructor, price)
- Variability of results based on device – mobile or desktop
We designed a mobile conjoint to present consumers (N= 1,245 yoga enthusiasts) with several alternative yoga classes including different combinations of these 5 variables. Consumers were asked to choose one offering, or none (if they were not interested in any of the options presented). This task was repeated several times to understand each consumer’s preference level for the various attributes.
Some of our key considerations in this study included the fact that mobile survey participation, while increasing, continues to lag overall. We wanted to know if better design could increase mobile participation and representation, and whether a discrete choice exercise produces comparable results and quality regardless of device. We wanted to know how these discrete choice exercises would render on smaller device screens and what impact we’d see on the data if we excluded mobile participants from the sample.
Key Takeaways
What we were able to take away from this study was that, at a high level when comparing mobile against desktop, consumers did not change their behavior based on price, although there is a group of consumers in this category that are price sensitive.
Discrete choice data provided a deeper look at opportunities and worked well on both mobile and desktop platforms. The consistency of data across platforms and a focus on a good experience for participants opens doors for improved representation among mobile audiences. There was a mobile preference among participants of 48.5% compared to 34.9% for desktop.
What we know is that participants expect mobile opportunities for research and many choice-based designs work well on smaller screens. At the same time, how many surveys are truly designed for a mobile audience? RelevantID is the leading digital fingerprinting technology in the industry and is widely used by both sample and full-service providers. Based on stats from their system, 40% of inbound traffic going into surveys is mobile. Essentially, by excluding mobile you are cutting your feasibility in half. Some providers are as high as 60% mobile traffic.
It is important to keep these factors in mind as we work together as an industry to further improve the participant experience in both mobile and desktop surveys. Learn more about the research process and the efforts of this important ROR. Watch a special webinar presentation of the findings here and download the deck from that NEXT presentation here.
