In our post-show wrap up article about The Quirks Event – East in Brooklyn, we mentioned JD Deitch’s talk Hidden Sampling Practices are Ruining Your Survey Data. Here’s How You Can Stop Them. The topic struck a chord with attendees, who asked a lot of questions after the presentation and even continued the conversation later in the day by approaching JD and our team. Sample quality is obviously an issue that needs to be revisited by the industry and addressed head on.

During the talk, JD covered quite a bit of ground including delving into these key questions:

  1. Where does sample come from?
    In this section of the talk, JD delves into performance marketing, real-time (or river) sample and partnerships with other sample companies. Online recruitment has vastly changed the way we acquire sample, driving down costs and driving up the possibility of fraud.
  2. How does it get to your survey?
    Simple email invitations eventually adopted new methods to, conceivably, allow users greater control using offer walls, routers and automation. But these methods still didn’t make users engaged and happy, so we have more work to do.
  3. What happens during fieldwork?
    Getting a survey into field is just the start of the user engagement journey. In a world where everyone is moving faster, things like bad survey design, technical issues, and manual field monitoring delays (like quotas that are met but not closed) leave respondents unhappy and unengaged. JD argues that we should be using every data point possible–including respondents’ own ratings of their experience–to stop bad experiences in their tracks.

As sample companies, we must to go beyond “fast and cheap” and start creating rewarding experiences for respondents at every step of the way. We expect to be covering sample quality in greater detail via blog articles and even an eBook in the coming months. Download the presentation slides here.

Find JD in April at the Insights Association NEXT Conference April 30-May 1 where he’s presenting Using Automation and AI to Shut the Door on Fraud in Market Research: When the Hispanic Mother of Two from Nebraska is a Russian Hacker