EVENT SURVEY
ttendee surveys are one of the most powerful tactics in the event manager’s arsenal. They can help you improve the content and structure of your event. They can keep you abreast of key issues facing your attendees and your industry. And they can reveal what your attendees really want from your event. But to get results like those, you have to be able to create a survey that your attendees will actually take, and that will provide meaningful responses. This has become increasingly difficult as more research companies emerge and people are bombarded with surveys both at home and at work. Skip Cox, president of Exhibit Surveys Inc., a trade show and event research firm in Red Bank, NJ, estimates that survey response rates are now about half what they were 15 to 20 years ago. Here are some tips and strategies from survey experts and event managers that will help you maximize the effectiveness of your surveys.
Survey Length When it comes to survey length, there is no right number of questions — you have to know your attendees. How passionate are they about your event and how much time are they willing to give you? Most experts agree that there is a direct correlation between survey length and response rate. “It’s safe to say that the shorter the survey, the higher the return,” Cox says. A long survey may also affect the quality of responses. Harris Schanhaut, manager of marketing events for American Express says, “When it’s too long, they whip through and leave parts incomplete.” While shorter is always safer, Kimberly Kee, president of Kee Consulting, a marketing-consulting company in Castle Rock, CO, says that if your attendees are vested in your event, you can go further and delve deeper. “You don’t want to waste people’s time, but this is not a random survey — these people have already qualified themselves by choosing to participate in your event,” she says. Each year, after the annual user conference for Hughes Network Systems Inc., one of Kee’s clients, Kee sends out a survey with 80 to 100 questions divided into eight to 10 sections — and gets a 40- to 50-percent response. Her secret? She appeals to the audience. “I tell them their responses are vital to the process of improving and enhancing future events,” she says. So how long can you go? According to Rob Aston, director of brand and communications at Hewlett Packard, there is also a correlation between the amount of time the participant has invested in the event and the length of the survey. He limits surveys for shorter events to 10 questions, and extends his surveys for multi-day events to 15 questions. Kee follows the same principle. She says a survey for an evening hospitality event should take three to four minutes, a survey for a day-long program should take up to 10 minutes, and a multi-day conference survey should take up to 15 minutes. Cox says another factor to consider is the profile of the respondent. He has found that specific types of respondents show distinct characteristics in the amount of time they are willing to invest in a survey. For instance, the response rate among high-level attendees, such as CEOs and CIOs, is higher when the survey is short, while other groups such as engineers and librarians typically invest more time. As for survey format, Cox says the length should be about the same whether the survey is done online, by hard copy, or by phone. However, online and phone surveys provide the surveyor the opportunity to use branching — customizing the questionnaire based on the respondents’ answers to previous questions. This shortens the survey by eliminating irrelevant questions and allows the surveyor to focus on questions that relate more directly to the respondent’s profile.
Question Content Event survey questions generally aim to evaluate three categories of information: attendee demographics, attendee satisfaction, and whether or not an event’s objectives were achieved. Attendee demographics: Surveys are a valuable tool for collecting demographic data about your attendees. Such data enables you to profile attendees in terms of background, job responsibility, buying influence, and size and type of company. You can use the information to identify trends in how the attendee base is changing year to year and to determine if sub-sets of attendees have distinct needs that you can address with specialized content or activities. For example, Schanhaut analyzes the job titles and responsibilities of his program registrants to make sure the planned content level is a match. When his attendee roster includes a large number of CEOs, he selects appropriate high-level sessions and content. Cox suggests that you collect demographic information during event registration. This enables you to integrate the data into your survey database. Then when you send a survey, you can link it to that information and you don’t have to ask the same questions twice. You can also use the demographic data to customize surveys to specific segments of attendees. Attendee satisfaction: Measuring your attendees’ satisfaction level can reveal what you need to do to improve your program, as well as any unnecessary investments. Kee asks attendees to evaluate every aspect of the event, including the registration process, fees, Web site, sessions, keynotes, food, accommodations, expo, and gifts. Sometimes the answers help you identify an area that isn’t as important to your attendees as you had originally assumed. For instance, one of Kee’s surveys revealed that her client was spending more money than necessary on attendee gifts. “We always worried about spending enough money on gifts, but our survey showed that their favorite gift was a simple laser pen,” she says. Kee even uses surveys to measure her measurement techniques. She asks, “How strongly do you agree with this statement: This survey gave me the opportunity to express everything that was on my mind.” Then she follows up with an open-ended question, such as, “What didn’t you get to express?” Achievement of objectives: “Make some of your questions measure whether you have achieved your marketing objectives, rather than just focusing on customer-satisfaction issues,” Cox says. Look back at your company’s major objectives for the program, such as accelerating the sales process, and ask questions that will reveal your success. For example, you might ask, “After attending the conference, are you more likely to use the product in the future?” You can also ask questions that measure perception and indicate whether you have communicated your message effectively. For example, ask, “How strongly do you agree with this statement: XYZ Co. listens to its customers?”
Question Format In general, there are three main types of questions: check-off questions, such as multiple choice, in which respondents choose from discrete options; scaled questions; and open-ended questions. Most surveys use a combination of these question types, depending on the type of information each question is trying to generate. Check-off questions: This type of question includes multiple choice, yes or no questions, and any other question that has distinct options for respondents to choose. According to Kee, this makes it easier for attendees to complete the survey and for you to tally the data and compare the answers. For instance, if you ask, “How did you find out about our event?” give choices such as: “Magazine advertisement,” “Direct-mail piece,” “Web site,” and “Through a colleague,” as well as an open-ended: “Other.” Scale: You can also scale questions to measure things like value, rank, satisfaction, or agreement. Kee likes to ask attendees to rate their agreement with specific statements on a scale of one to five — with one meaning “strongly disagree” and five meaning “strongly agree.” For example, consider the question: “Did participating in the conference increase your likelihood of working with XYZ Co. in the future?” A “no” response could mean, “No, I’m not going to purchase from you.” Or it could mean, “No, I was definitely going to purchase from you either way.” A better way to phrase this question would be to ask the attendees to rate their agreement with specific statements on a scale of one to five. Then you could ask for a response to two statements: “Prior to attending the event I was planning to purchase from XYZ,” and “After attending the conference, I plan to purchase from XYZ.” Open-ended: Always leave an opportunity for write-in answers and suggestions — especially when you don’t know what the answers are going to be or are looking for suggestions. “When you give your customers a chance to speak, even the shy ones will give you valuable insights,” Schanhaut says. Cox includes open-ended questions to humanize the results and to get feedback. “The open-ended question gives you the why,” he says. For example, if he asks a scaled question like “Rate the value you received from this event,” he often follows up with the open-ended question: “Why did you give it that rating?” Kee suggests leaving an area at the end of every section to add comments. These responses often reveal a mood you don’t get from the other responses, indicating the attendees’ overall level of satisfaction with the event. They can also uncover problems. One of Kee’s surveys revealed that while attendees loved the venue, many had problems with hotel billing.
Testing Questions If any of your questions can be interpreted in more than one way, you’re not going to get valid responses. Sandra LePedis, vice president of marketing for RSA Security Inc., avoids this pitfall by having several people read the questionnaire before she sends it out. For instance, the questions, “Was the keynote up to your expectations?” or “Did you like the keynote,” could be interpreted in a variety of ways. But “Did you find the content of the keynote to be valuable and relevant?” is much more specific. Cox says the best way to test questions is to give the survey to a group of attendees to make sure they complete it without confusion.
Incentives Most professionals who have conducted survey tests agree that an incentive encourages participation. “We use incentives 98 percent of the time,” Cox says. However, selecting the right incentive can be difficult. The trick is knowing your audience. Exhibit Surveys continually does research to determine what level of incentive is most effective. For example, a couple of years ago, the company tested an incentive by randomly splitting the survey sample into two groups. It sent out the same letter to each group, inviting respondents to have a cup of coffee while they filled out the questionnaire. Then it sent $1 to one half, and $2 to the other half. There was no significant difference in response rate between the two groups. The value of your incentive depends largely on the number of responses you need and on the value of the responses to you. For instance, if you have a small event and need a large number of responses, your incentive may have to be more expensive than if you have a large number of attendees and require only a small percentage sample. Aston likes to select an item with an approximate $25 to $30 value that appeals to his audience — like plug-in computer memory. “A key chain wouldn’t get it,” he says. Free registration for next year’s event makes a popular prize. And while it’s valuable to attendees, it’s relatively inexpensive for you. Raffles are another option that usually require a more valuable prize — possibly as much as $1,000 value, says Aston — but you only need one. Before you dangle a carrot in front of your attendees, weigh the value of the increased response rate against the cost of the incentive. “If you gave everyone a $20 gift certificate to Amazon.com, you’d boost the response rate, but then it’s not practical to do the research,” Cox says. Sometimes the most compelling incentive is an appeal to the audience that its opinion is valuable. Cox has seen a direct correlation between communicating this point and response rate. “Attendees spent the time and money to go to the event, and we want their opinions about it. That’s the primary reason our response rates are high,” he says.
Timing As you consider when to conduct an attendee survey, think about what type of information you want to collect. Surveys before the event, during the event, and after the event can each serve different purposes. You may even want to use all three. Before the event: Most pre-event surveys are integrated into the event’s online registration interface. In addition to capturing demographic data, including names, titles, and product interest, these surveys can reveal what the audience expects from your event. LePedis uses this opportunity to double-check conference content. She asks her registrants which sessions, topics, and issues they are most interested in, and then adjusts her program accordingly — filling holes and adding more sessions to support the hottest topics. Another valuable trick is to use a pre-event survey to learn how people are finding out about your event. This reveals which parts of your promotional strategy are working, and which parts are falling short. At the event: During an event, Aston distributes hard-copy surveys to attendees to get feedback on session length, speakers, content, and the value of sessions to attendees. This ensures that details are captured before attendees have time to forget them. For evaluations of individual conference sessions, LePedis and others say the best time to strike is immediately after the activity — before the day’s activities turn individual sessions into one big blur. After the event: Evaluate the experience as soon as possible after the event. Your data will be most accurate (and your response rate highest) if you reach your attendees while they still remember your event clearly. Exhibit Surveys finds that response to surveys sent two weeks after a show is typically about 5-percent higher than surveys sent four weeks after a show. According to Kee, the ideal time frame is one to two weeks after the event. This gives people time to return to their offices and catch up, and to process the event mentally. But it’s soon enough that they haven’t forgotten or lost interest. Kee has found that this time frame produces the highest response rate as well as the most detailed responses. When surveys go out later, she receives less commentary and more skipped questions.
Consistency Make each new event survey consistent with surveys you’ve used in the past. This creates the opportunity to compare event surveys from different years and from different events. Terry Reynolds, who is in charge of business operations and planning at Intel Corp., accomplishes this by repeating specific questions on all surveys. The result is an apples-to-apples comparison that reveals which events were successful and why. Reynolds says the most important questions for her are those that pertain to how people view the Intel brand, such as “What does the Intel brand represent?” Intel uses those same questions on surveys that evaluate other types of marketing programs. The company can then compare the results of its events with results from other types of marketing to determine where its dollars are best spent. The benefit? “When wrangling for resources and budgets, you’re playing on an even field,” Reynolds says. In the end, survey data will help you understand your audience better, identify industry issues, and refine future events. It will also enable you to document your events’ contributions to the company, which makes a compelling case for job security and continued funding. As Reynolds says, “What gets measured gets done… or paid for.”
Lisa Goell Sinicki is a free-lance writer in Peaks Island, ME.