The Panelist Dilemma: Quality Isn’t Expensive, It’s Priceless


Illustrating the parallel between hiring qualified staff and recruiting top-notch panelists.

LOS ANGELES, March 15, 2016 — Many years ago, while at a former company, I was asked to fill several key positions in the Operations department. The company was growing rapidly and several folks on staff were headed on a fast-moving train to a place some of you are all too familiar with: Burnt-Out City. As a manager, I knew I had to alleviate the pressure our team felt and FAST. Unfortunately, I was given a meager budget and a very tight schedule. As such, I failed to do my due diligence and I made a poor hiring decision that still haunts me to this day.


If you know me personally, you know I am the type of person that likes to cross her t’s and dot her i’s, so the situation made me uncomfortable, to say the least. Timing was critical on one particular hire and I decided to extend an offer based on my interview with him. Only the interview. No reference or background checks. This particular candidate made a great impression during the interview process; he said all the right things and he was a very low-cost resource for the company. Successful outcome? Think again.


Fast forward to a year later, and I was being interviewed by an LA County detective who would be pursuing that employee for embezzlement charges. That’s right – my hire stole money from the company. I was mortified, angry and disappointed that I didn’t do my due diligence. Had I done a background check, I would have found out that this was not his first felony. I made a hasty decision, and our company paid for it. This experience taught me that proper due diligence and preparation are key ingredients for success. I’ve since hired dozens of people with much more positive outcomes. The best employees are the people who come to me through a referral, have a background check, and possess work experience that I can count on. Ultimately, the big takeaway for me is that reputation is everything and having the right people on staff will make or break your company.


So you are probably wondering, “What does hiring staff have to do with sourcing high quality panels?” When you are in the insights business, a well-performing survey panelist is another representative of the company. Once you invite a respondent into your survey and they qualify for the study, they now directly represent your company, your brand and the choices your company makes…just like your employees. It’s that simple and yet over the last few years, the sample space continues to become more and more commoditized. Pricing compression is at an all-time high and quality is often considered as ”nice-to-have.”


Let’s be really clear: as a buyer of sample, you can have good OR you can have cheap, but I don’t think you can have both. Of course, I don’t want to paint all buyers with the same brush. There are many buyers that I work with that talk about the importance of quality and exercise a great level of discipline and protocol when vetting sampling firms. I want to give these folks a big shout-out and commend them for their focus! I wish everyone was like you. Unfortunately, I still see many companies procuring sample without proper vetting or (respondent) validation, often awarding work to the lowest-cost supplier. In my experience, cheaper is never better. It isn’t “just as good” either. You get what you pay for. Anyone that tells you that you are getting the same level of due diligence and quality (at a discounted price) is being disingenuous.


Here’s why: there is a REAL cost to recruiting and managing a survey panel. I’ve had countless clients attempt to build a panel on their own only to realize that they have made an expensive, regret-filled decision. It isn’t an easy business if you want to do it right. In a perfect world, I’d love to provide the best and least expensive sample in our industry, but that is pure craziness and fiction. I cannot give you both.


The “best quality” means that I am committed to hiring the most seasoned staff and using the best tools to deliver balanced, clean sample. I can achieve this goal because I have the experience. What I can’t do is offer all of those things for bargain-basement pricing.


Here’s why a quality sample costs more: 


  • Experienced and vetted talent commands a higher salary
  • Efficient tools require significant investment and development
  • Balanced and clean sample requires literally dozens of methods designed to test honesty, attention and legitimacy 
  • Fraudulent activity is rampant and requires significant resources to thwart this behavior

I often liken online panel fraud to a computer virus. Just when you have solved for a particular strain, a new, more sophisticated strain of fraud will surface and catch you off-guard. Fraud prevention requires laser focus; none of us in the sample business have the luxury to be complacent or arrogant about this issue. Several layers are required; some of these methods are programmatic, but the art of implementation cannot be programmed. Our business requires human brainpower and that, my friends, has value not available at the discount store.


My advice to buyers of sample: take your time and introduce a formalized process at your company for vetting your sample suppliers and their respective panels. Don’t rely on panel books or a simple interview to guide your decision to buy sample. Ask the tough questions. Demand to see how these companies manage their panels and produce quality samples. Above all else, remember – quality isn’t expensive, it’s priceless.




About Lisa Wilding-Brown:
Lisa Wilding-Brown leads Global Operations for Innovate and has over 13 years of experience in the Market Research space. She started her career at Harris Interactive on the client services team helping clients migrate their traditional telephone studies to online research. Later, Wilding-Brown moved to the panel management team where she was responsible for panel recruitment and retention of the Harris Poll Online, the industry’s first premier online research panel. In 2009, Lisa joined uSamp, an online global sample supplier. As a member of uSamp’s executive team, Lisa was responsible for many of the company’s technology initiatives including the firm’s mobile business division and online panels. During her tenure, Lisa served as the primary thought leader for developing the company’s mobile strategies, methodologies, quality protocols and API solutions. Now at Innovate, Lisa and the Innovate team have set out to redefine the user engagement model; their new twist on panel management is staving off typical attrition challenges which in turn provides clients with greater reach and improved quality.


About Innovate:
Real People. Quality Data.™ Innovate is a global online sampling firm, generating high-quality data from engaged panelists. Founded by Matt Dusig, Gregg Lavin and George Llorens, Innovate provides 24/7 client service to thousands of market researchers and research departments around the world. Innovate pioneered Human-Powered Sampling™ which promotes responsive communication for client satisfaction and created the First Class Panel™ which is a heavily-screened, profiled and engaged audience of survey respondents. Follow Innovate on LinkedIn, Facebook,Twitter or Google+.