Don’t Fear The Research: A PR Checklist for Designing Killer Surveys
You’ve heard it from clients and journalists alike: “we’re going to need some data.” You know it’s coming — a survey project looms on the horizon.
PR folk, by trade, excel at words and ideas. We’re trained to wordsmith our big ideas into neat packages, ready for consumption. Data are, by contrast, messy, and the thought of having to dig around for inspiration in an Excel spreadsheet can be enough to send the most seasoned among us into a fit of teeth-gnashing angst.
But with some careful preparation, and by following a few simple questionnaire design guidelines, you can steer a survey project from stormy waters to calmer seas.
1) Envision your overarching research question — and your results.
You’ve spoken with your client, you’ve done your research, and your team brainstorm is on the calendar. Time to get cooking on some killer question ideas, pop them in a Google Doc and whittle down the list later. Just like a story brainstorm, right?
Not so fast. First, ask yourself and your team the following: if you boiled down your survey to a single question, what would it be? Is it a question that anyone’s asked before? More importantly, will reporters covering your industry find the answer to this question interesting, no matter what that answer is.
This is where survey design parts ways with story ideation. In the latter, it’s common to collect proof points and connect them to demonstrate a larger trend. But surveys work in the opposite way. You need to start big with a question that hasn’t been asked before. From there, you’ll branch out, asking sub-questions to map out your concept and ensure you’re fully answering the primary question.
Hint: If you think you know the answer to that big research question already and the alternative answers aren’t all that interesting to you, consider a new question.
2) Put yourself in your survey respondent’s shoes.
So you’ve got your overarching research question, you’ve mapped out what you’ll need to answer it, and you’re brainstorming your questionnaire. The sky’s the limit: let’s dive into the depths of our participants’ earliest childhood memories, or probe their knowledge of the frontiers of scientific innovation, or maybe let’s see if we can coax out some bold new opinions on the merits of SQL vs. NoSQL databases.
Time out. This is another point in your survey design process where it’s critical to pause, put yourself in your respondent’s shoes, and ask yourself the following:
- For questions involving memory or recall, will respondents be able to accurately remember what you’re asking them?
- For questions involving attitude, is it likely that respondents will feel strongly about the subject? If they don’t feel the way you expect they will, is that a deal-breaker?
- For questions involving comprehension, is it a fair assumption that the respondent will intimately understand the topic or trend?
Hint: This is a good time to loop in a co-worker that isn’t on the account. Even if they don’t understand every question, it’s always good to have a fresh set of eyes on subject matter you’ve been (a little too) close to, and they’ll serve as a good litmus test for identifying any questions that made sense in your head… and less so on paper.
3) Arrange everything in the right order.
The best PR pitches engage journalists with a clear narrative, and the same holds true for engaging respondents with a survey. Even if your questions make sense individually, poor ordering can create a confusing experience for respondents — and lead to disappointing data.
A few pointers on smart ordering:
- Your first questions should be relevant and easy. Don’t come out of the gate swinging with questions that are too complex or specific. Start off slow and give respondents a chance to get comfortable with the subject matter.
- Group similar questions into modules. Since your survey probably won’t consist of 20 isolated questions with no bearing on each other, try to arrange questions about similar topics into groups, and make clear transitions with headings and numbering. (This will also make your analysis a whole lot easier later on.)
- Place sensitive questions toward the end. If you’re surveying your respondents about personal topics, don’t ask anything too sensitive right off the bat. Let them ease into it — and be sure to offer context for the overall purpose of the survey, as well as your assurances that their responses will be kept confidential and anonymous.
- Place demographic questions at the very end. These questions are usually the easiest for respondents to answer. They’re also the most boring. It’s safe to assume your response data for these will be accurate, so save them for last.
4) Take the whole survey yourself (and make a co-worker take it).
One of the biggest pitfalls of a survey is its length. After you’ve honed your questions and arranged them in the right order, be sure to take the survey yourself. Then have a co-worker take it. Afterward, ask: Was that ridiculously long? Was it tempting to start just clicking through as fast as possible?
If the honest answer to either of those questions is “yes,” you can be sure your respondents will agree — and you should consider shortening the survey.
Many professional sampling services will allow you to ask as many questions as you want without upping the cost, but it’s your job to step in: if Question 72 needs to be asked, it needs to be asked sooner. Most surveys don’t need to be longer than 20-25 questions.
The quality of your data will suffer for questions asked toward the end of a long survey, in two major ways:
- Satisficing effect: Respondents will start picking the first available answer that is “good enough” to satisfy the requirement of the question, without evaluating whether it’s the most accurate.
- Primacy effect: Respondents will start picking the “top of the list” of answer choices to try to get the survey over with, no matter whether the answer is true at all.
Hint: Long surveys mean more data for you to comb through, too. Don’t make life harder on yourself than it needs to be.
Ready for takeoff
The day has come: it’s time to put your survey into the field and let it soar. Your team is on the edge of its collective seat with excitement, and your other co-workers are relieved that they won’t be subjected to yet another test run.
Now’s a good time to remind your client, your team, and most importantly, yourself, that you ultimately can’t control the results — you can only control the quality of the questions. But if you’ve made it through this checklist, you should emerge with a pretty good story to tell with your data — even if it isn’t the one you expected.