Sampling in the Asia Pacific Region Q&A

APAC has really been seen as a hot bed for quality issues in our industry. What do you think the main drivers are for those quality issues in the APAC region?

Jeff Gerken: China is becoming a hot bed for fraud, with panel houses and panel farms. And one of the main reasons behind this is that the hourly wage can be anywhere from $1-$1.50.

If you could sit behind a computer in an air conditioned room and take a B2B study that pays $10-15 for a 15-20 minute survey, it would look pretty good compared to standard hourly wages.

How about you Rie? Any thoughts to share and expand upon what Jeff just said there?

Rie Nagai: We’ve all heard of the fraud in China, but what is it all about? The security measures we take really need to be properly implemented when running research in Asia. They are professional fraudsters. Like Jeff mentioned, they’re being paid much more than they might otherwise make to complete these surveys. 

I think there’s a lot of economic motivation, and it is such an unknown area of the world for a lot of U.S.-based and European-based researchers. The challenge as researchers is that we’re asked to do global studies in multiple geo locations and multiple markets. It’s difficult to have the appropriate internal resources for quality control and assurance for these markets – a lot of clients we work with are dealing with this.

Let’s move to the next question. InnovateMR survey audiences span multiple countries in APAC. What are some of the key similarities and differences within those markets for survey research and how do they impact delivery?

Rie Nagai: From my perspective the APAC region can be divided into two areas. There’s the developed markets like Japan or South Korea where the Internet penetration is 100%. With those markets, online methodology can reach most of the audiences, even the tougher ones.

On the other hand there are developing markets like Indonesia, Thailand, and the Philippines where the population is growing, but the Internet penetration is still quite low. Lower than a lot of people think. Here, we still need to apply mixed methodology, and what we do really depends on what audience we’re trying to reach. As a result, B2B and elderly audiences are easier to approach in countries like Korea or Japan, but it could be very hard in the Philippines, Vietnam, or Thailand.

Additionally, it’s easy to forget how small a market countries like Singapore, Hong Kong, or even Australia actually are. The same sampling plan wouldn’t really apply compared to the U.S., UK or other major Western countries. So, we really need to be aware of that.

What are some key challenges for designing effective surveys in the APAC region? 

Rie Nagai: One of the major challenges is the difference in sensitive topics in APAC. It might not come across as sensitive in countries like the US or UK, but healthcare projects are a great example. A lot of the survey respondents in Japan or Korea, for example, take such questions quite personally. So, response rates could be a lot lower than you might expect.

Political questions in countries like China, Hong Kong, Singapore and Taiwan can be very tricky as well. Panel providers can receive a lot of complaints by asking the wrong questions. Certain domains can potentially get banned if the survey includes political content.

Religion is the same, especially in Indonesia and Malaysia, or other predominantly Muslim countries. Alcohol-related content would, for example, be considered insensitive. That type of cultural sensitivity can be challenging for companies not accustomed to these differences.

Jeff Gerken: There are also the national holidays, which vary from those celebrated in the U.S. or UK. For example, Christmas is a relatively minor holiday in APAC, but New Year’s celebrations can shut down a country for a week or more.

Internet bandwidth is a big issue as well. When clients ask how they should write their questionnaires, more than 10 minutes is too long. Because most people will be doing these on a mobile device, and with the internet bandwidth variability going from .5 MBs up to 15 MBs within a 10 minute span, long surveys are a problem. You will see a lot of drop off.

Especially in countries like China where a lot of people are on VPNs that go in and out, it’s something to really take into consideration when you’re writing those surveys. You also have to keep everything mobile friendly because that’s how a lot of people access surveys, even if it’s not intended to be a mobile survey. 

Any thoughts on the various websites that have been banned in China? How does that affect our ability to engage those participants?

Rie Nagai: Being a communist country, China has banned more than 5,000 websites including Facebook, Twitter, Google, YouTube, and a lot of other very large sites. These sites are not accessible except via VPN, so adding YouTube hosted video to content will no work properly in China. Additionally, the social media and news media usage rates won’t be proportional to other market trends.

Jeff Gerken: There are a lot of companies that claim to do sample in APAC, but unfortunately there’s a lot of aggregating going on. There’s a lot of sample being pulled off of exchanges, and people and companies that don’t really understand where the sample is coming from. And so it’s really important that you work with a provider that understands APAC. Otherwise you’re going to get this river of sample coming in.

Trying to enable fraud mitigation factors, and trying to figure out what’s real and what’s not can be really tough. Innovate has done a fantastic job of bringing people like myself and Rie in, who understand the industry and work with in-country providers. We can have a one-on-one conversation with them and our vetting process at Innovate is intense. What we expect from our providers and what we put them through to make sure that what they’re giving us is real, and is going to benefit our clients, is exceptionally rigid.