We’ve really welcomed the surge in discussions of late around the topic of online data quality. It’s been an issue for many years -are people who they say they are? Are they even genuine humans? And it gets far worse with B2B…at the end of the day I’ve never had a single decent answer to the question “why are senior B2B execs, who get paid high 5, if not 6 figure salaries, doing surveys for points the equivalent of 50p at a time?” I know the answer: they’re not!
There’s a lot of what I’m calling ‘disruptors’ cropping up offering verified respondents or ‘expert networks’ that are offering much higher levels of assurance and with it, CPI’s that in many cases are now in excess of CATI. But having used a number of them, paying high doesn’t guarantee a genuine respondent. This has improved the data but when we’ve used our aggressive screening we’re still seeing well over 50% of data being removed. It’s not good enough. There needs to be more transparency about who we are paying to speak to.
Paying high doesn’t guarantee a genuine respondent.
There’s no substitute for actually speaking to a human for confirming you’re actually speaking to a human…but online is needed so we can continue to moan, or we can look how to combat the ongoing issues. For the last two years we’ve adopted some additional techniques to try and cut through the noise and elevate data quality.
Our approach balances the scale of online data collection with the rigour of human validation. We deploy all the usual raft of algorithms and AI machine learning mechanisms to remove bots, speeders, repetition which helps remove the top layer, but then we get to our aggressive screening. We take the clients q’aire and take another look at it. How obvious is it? We look at it like a game of poker- seeking early tells and whether they reveal their hand too soon.
Our approach balances the scale of online data collection with the rigour of human validation.
A great example for B2B is at the very start, before we even get into the standard screening is to ask two open-ended questions. “What is your job title?”, and “give me a bit of detail about your job role – what are your key responsibilities?”. Not only does this check for their basic level of engagement, but whatever or whoever is completing the survey has set their stall out before anything about the survey is revealed or standard screening is asked. It acts as a reference point for manual human-based checks against the rest of the data, which can be done almost real time so adjustments in quotas can be made whilst in field. When you then ask the standard single-coded questions around job role, responsibilities etc that you typically screen on, you can quickly identify contradictions and conflicts. You can’t confirm you’re a customer Service Adviser and then say you’ve end decision making status for the companies IT infrastructure!
I’ve never had a single decent answer to the question: ‘Why are senior B2B execs… doing surveys for points the equivalent of 50p at a time?’ I know the answer: they’re not!
There’s a number of other questions we include to assess their knowledge & experience of the topic – from simple red herrings to more complex lines of questioning. However, these measures are not an exact science. It does often reduce feasibility as we remove much higher percentages of completes that would otherwise ‘get through’. When combined with the algorithms and AI based quality checks what you end up with, we believe, is first and foremost genuine human respondents, but also more assurance they qualify. Failing that; speak to them via CATI or F2F!
.svg)
