> HOME

> ABOUT US

> CLIENT LIST

> UWA LIBRARY

> SITE MAP

> The Five Rules for Reliable Marketing Research
> Reliable Marketing Research: Process Diagram (PDF)
> Whether to Use Qualitative or Quantitative Research to Answer a Marketing Question

PRINT-READY FORMAT (PDF)    

The purpose of customer research is to develop new knowledge about a market. In order to do this, surveys must be designed to gather accurate, reliable data about customer preferences and satisfaction. Then, the collected data must be tabulated so that it reveals information and analyzed to produce new insights.

Marketing Research Produces New Knowledge

The amount of information and insight that you can extract from marketing research will determine the success or failure of your product or service. Following these five rules for reliable marketing research will dramatically improve your probability of success. Perhaps most importantly, these rules will help you avoid expensive failures.

By following these five rules for reliable marketing research, you will gain new knowledge about the markets for your product or service:

1. Speak to the right people.

2. Use only customer language.

3. Use semantic scales.

4. Avoid artificial choices.

5. Analyze your research data.

Accurate, reliable information is the key to marketing success. By following these rules for reliable marketing research, you can uncover the information that will move your business ahead. By understanding your customer well, you can increase your company's competitiveness.

Rule #1: Speak to the Right People

Interview the Right People When Conducting Customer Research

This is an important area of marketing research that often gets overlooked. For example, in the 2004 presidential election, the reported results from the "Exit Polls" were extremely inaccurate. It turns out that the interviewers spoke with very few men, and thus, the results were biased by a predominantly female respondent pool. Since voting patterns can vary by gender, it would have been prudent to speak with a representative sample of men and women. Alternatively, in situations where you are not able to reach a representative sample of your universe (perhaps in this example because men were less willing to be interviewed or because fewer men than women voted during the part of the day when the interviews were conducted), you can weight the responses to reflect a representative sample of all voters.

Sometimes there are instances when you do not want to speak with the whole universe of respondents. For marketing reasons it may be important to speak with a specific subset of the whole universe of possible respondents. Examples of ways that you might want to define respondent groups include:

> Those who have positive purchase interest in a concept

> Heavy users of a particular category

> Certain corporate decision makers for a business product or service

WAYS TO CONTROL THE RESPONDENT GROUP

1) Draw the sample for the research so that it includes the proper target audience.

2) Add a screener to the beginning of the research that will drop respondents who do not meet your respondent profile.

3) During research keep monitoring to make sure that a representative sample has been interviewed.

4) Verify at the end of the research that the respondent group is composed of the people with whom you wanted to speak.

Rule #2: Use Only Customer Language

Use Customer Language and Address Only Those Issues That Are Relevant to Customers

In order to gather accurate and reliable information, the questions used in customer research need to be clearly understood by the respondents. It is especially important that every respondent understand each question in the same way. When questions convey different ideas to different respondents, or when they are interpreted differently by different respondents, then the research answers lose all significance.

Just as important as using customer language is addressing only those issues that are relevant to customers when conducting customer research. Ask customers about benefits or about how it feels to be a user of a product or service. These are topics of great importance to customers. Customers have opinions about, and an understanding of, benefits and how they feel. Thus, they are can reliably answer questions about these topics.

It is important to avoid overly detailed or very technical questions in customer research. This is a very common error. Overly detailed and very technical questions are usually confusing and always tedious. Because the answers to these questions lack meaning to the customer, the research results from these questions fail to produce reliable learning.

Rule #3: Use Semantic Scales

Use Semantic, Rather Than Numeric, Scales in Order to Gather Consistent Answers from Respondents.

In quantitative research you have a choice between two different kinds close-ended responses. You can give people an opportunity to respond with words (semantic response) or with numbers (numeric response). With semantic responses you are more likely to generate answers that have the same meaning to all your respondents and thus provide the most reliable research data.

One of the classic marketing research issues is the measurement of "How well-liked is a product, service, or an idea?" There are two approaches for collecting this quantitative data:

Alternative 1: Would you rate this product, service, or idea: Excellent, Very Good, Good, Fair, or Poor?

Alternative 2: On a scale of 1-10, with one being the lowest and ten being the highest, how would you rate this product, service, or idea?

Comprehensive testing has shown that semantic scales are much more likely to generate consistent responses than numeric scales do. In fact, the semantic differential on the scale in Alternative 1 has been proven to be equal steps that correspond to 5, 4, 3, 2 and 1, where 5 is Excellent and 1 is Poor. Thus, different customers respond in a consistent manner to this scale.

On the other hand, different customers often respond differently when using a numeric scale. For example, many respondents equate "good" with "average" or the number "5". However, based on their school experience, another large group of respondents equates "good" with a "C" grade (70%-80% correct) or a "7-8" rating on a 10 point scale. Mixing both types of responses interferes with learning about customer evaluations of a product, service, or idea.

The bottom line: Semantic scales are easier for respondents to understand. Semantic scales generate more consistent results. Thus, research using semantic scales produces more reliable customer information.

Rule #4: Avoid Artificial Choices

Avoid Research Induced Responses That Do Not Reflect Actual Customer Opinions

Research respondents will answer whatever questions they are asked. Respondents have a desire to please the interviewer. More importantly, respondents have a need to give an answer that appears competent. Particularly in business situations, respondents are prone to answer what they think they are supposed to say. The more that questions stray from reality, the greater the occurrence of research-induced responses.

Asking customers to make artificial choices is a very common source of mistakes from customer research. For example, surveys of transportation company customers often ask "Which is worse: a late shipment or a damaged one?" Shippers are both able and willing to answer survey questions like this one; however, their answers are meaningless. In real life, shippers don't have to make this trade-off. Their shipments are delivered on time and without damage or they switch suppliers. These artificial choices resemble questions from the teen social game "Would You Rather" as in "Would you rather wreck your dad's car or have a bad prom date?" The question is fine for amusement, but not much else.

Measurement can convey a false perception of certainty. Just because the answers to a survey question can be counted does not insure that those answers convey correct information. Asking customers to make choices that they do not have to make in real life usually produces inaccurate and invalid information.

In order to avoid this problem, think about your research survey as a conversation with a customer. Would that customer agree that your questions reflect how he/she makes decisions? Would the questions and choices seem reasonable? Do the choices addressed by your questionnaire reflect the issues he/she thinks about?

Rule #5: Analyze Your Research Data

Learn Things That Customers Cannot Directly Tell You

People who only consider topline results or the answers to a few specific questions from customer research miss at least 75% of the value. Customer research results need to be analyzed and interpreted. Also, it is important to combine company data with customer research data in order to learn more.

Analysis is important because some marketing questions cannot be answered directly by customers. For example, customers are often not aware of all the factors that create brand loyalty. Thus, asking them directly to list why they are loyal to a brand will elicit only partially complete answers. An analysis of buying behavior versus different customer experiences will often reveal other important brand loyalty factors.

It is particularly important to develop a tabulation plan for analyzing customer research data before interviewing starts. This thinking beforehand will insure that all the questions that you need to have answered will be included in the research. Also, thinking about available company data in advance will often determine what additional data would be most helpful.

All material copyright © 1995-2009 by Urban Wallace Associates.