Public Renewables Partnership, PRP Conducting market research

What not to do: five popular mistakes

What's already known
General preferences for renewables
Willingness-to-pay
Market Segmentation

What to ask for in the RFP
What to look for in vendor staff and experience
What to look for in a vendor's standards and practices
What the vendor should do before the study: Preparation
What the vendor should do during the study: Execution
What the vendor should do after the study: Deliverables

Market research

What do your customers think about renewables? Are they willing to pay more for renewable resources? Do they prefer wind to solar? Market research – finding out what your customers think, want, and need – should be an important part of any utility's renewable planning process.

Here we provide information to help you get started down the market research path. We start by describing the five most popular mistakes, utilities make in market research – so you can avoid these common pitfalls. We then review and summarize what's already known, so you can sharpen your research questions so as to provide new knowledge. The market research process (link to below) can be quite complex, so we include a flow chart laying out all the steps. Most utilities will outsource some or all of their market research, so we provide detailed instructions on how to work with market research firms (link to below). Finally, we provide model survey forms (link to below) you can edit and modify to meet your needs.

What not to do: five popular mistakes

The most important part of market research–and the one most often done poorly–is careful thinking up-front about what you hope to find out and how you plan to use this information. Much–perhaps most – of utility market research $ is spent poorly. Here are our top five market research mistakes:

Looking for certainty in an uncertain world.
You will never have perfect or complete knowledge about your customers. Preferences, beliefs, attitudes – these are all not only difficult to define and pin down, but continually changing over time. Don't waste time and money in pursuit of unrealistic precision.
Doing market research as a way to avoid making difficult decision(s) about a new consumer product, a generation investment, or a utility policy. It's always a temptation to ‘do more research,' when what's really needed is a decision. One way to solve this problem is to assume success: that is, assume that your market research succeeds and that you're able to come back in six months with ‘the answer.' Will you now be in a significantly better position to make a decision? Or does the problem lie elsewhere?
Documenting what's already known. Many utility market research efforts have as goals determining their customers' willingness to pay, or documenting their overall attitudes towards and preferences for renewables. However these questions have been researched to death already, and the results don't differ significantly by region.
Using research to justify, not to learn. If you're looking for evidence to support a particular position, don't undertake market research unless you're willing to be shown to be wrong.
Not meeting decision-makers' needs. Market research should help utility decision-makers make better decisions. What do these decision-makers need to know? Will your planned market research provide that knowledge?

If you've satisfied yourself that you're not making any of these mistakes, then you're off to a fine start.

What's already known

Don't waste your time and money reinventing the wheel. Much is already known about consumers and renewables. Here we summarize what's already well-documented and clear. As we show, there is already abundant evidence on:

generic preferences for renewables
stated willingness-to-pay for renewables
market segments most likely to buy a green energy product

General preferences for renewables

One of the clearest and most consistent findings from energy-related market research is consumers' preference for renewables and efficiency over other energy sources. A comprehensive review of the energy market research literature concluded:

Trends in public opinion polls have also shown long-standing preferences for renewables and efficiency over other energy sources and usage patterns. Majorities of respondents to national polls dating back to 1979 prefer renewable energy and energy efficiency over other alternatives when cost or price are not mentioned. The pattern of preferences for using renewables to supply energy and energy efficiency to decrease demand has been consistent in the poll data for 20 years. This is one of the strongest patterns identified in all of the national poll data on energy and the environment.

There is no evidence that these results differ markedly by region: In other words, there's little reason for you to waste your time and money doing market research in your region or location that seeks to document overall preferences for renewables, as you already know what you'll find: consumers prefer renewables.

It's all too common, unfortunately, that utility decision-makers think that their customers' generic preferences are different. Utility researchers, therefore, may find themselves in the position of having to document what's already known. There are two ways to respond with this situation: first, try to educate the decision-maker that such research is unnecessary; or second, go ahead and do the research but write the survey questions so as to yield useful information on, for example, preferences considering cost.

That said, note that the results above are for polls in which cost or price are not mentioned. Consumer preferences when costs are considered are less well documented, and are certainly an appropriate area for market research.

Willingness-to-pay

When considering a green pricing program, utilities almost invariably start by doing willingness-to-pay research. Here again, such research is well-intentioned but is also a waste of time and money. Numerous utilities have already done such research, and the results are clear, well-documented, and do not vary significantly by region. (The results are misleading as well, as discussed below).

Insightful researchers at NREL pulled together results from 14 different utility willingness-to-pay studies. The results were amazingly consistent. As shown in the figure below, the data points are well-represented by the curve. (This figure is taken directly from the report). Given the variety in populations surveyed, survey question wording, and survey methodology; the consistency of the results is incredible.

What does this mean for your market research? As the report authors note, "given the coverage of the research, it is likely that any market survey asking residential customers about willingness-to-pay for power from renewable sources will obtain results similar to those represented by the curve." In other words, if you want to know what percentage of your customers will say that they will be willing to pay X amount additional per month for renewable energy, you don't have to actually ask them. Instead, just look at the curve. Consumers' stated willingness-to-pay is well-documented, and putting a few more points on the graph is not a good use of scarce market research resources.

There's another, and even more compelling, reason not to do market research on willingness-to-pay: the evidence suggests that there's little match between stated and actual willingness-to-pay. There are over 100 green pricing programs in place in the U.S., and most show participation rates of less than 5 percent – one percent is typical. This is in sharp contrast to the willingness-to-pay research, which consistently finds that over fifty percent of interviewees say they would pay a premium for renewable electricity.

Market Segmentation

A fair bit of work has already been done on market segmentation of the green energy market. A large survey of green energy buyers and nonbuyers across North America was undertaken in 2001 and 2002. This survey took data from 2,800 residential electricity users and is the most comprehensive and useful to date. The survey asked participants a number of questions related to political beliefs, views on environmental issues, and other possible indicators of green buying behavior, as well as standard demographic indicators. The results were then broken down into green buyers and nonbuyers. Results from this survey confirm some commonly held beliefs about green buyers but also show that a few beliefs are incorrect.

Indicators showing the greatest difference between buyers and nonbuyers were political persuasion and support of environmental groups. Almost two-thirds of green buyers identified themselves as "liberal" (as opposed to just 38 percent of nonbuyers), and just under two-thirds of green buyers were supporting an environmental group (as opposed to just 28 percent of nonbuyers). Education level was also a strong predictor, with green buyers much more likely to hold advanced degrees. Green buyers are also more likely to be politically active.

As the study's authors noted, environmental activism was a surprisingly strong indicator. "(We were) not surprised to find that participants were more likely to have knowledge of, and interest in, the environment. We were, however, surprised by how powerful an indicator environmentalism proved to be. And when we asked participants to explain why they joined the green energy program, they most often gave explanations couched in environmentalism: ‘It's an opportunity to do something for the environment'; ‘Because I think we should protect the environment and it's convenient'; and simply, ‘For environmental reasons.'" Also notable is what didn't show up as significant. Neither income nor gender showed much variation between buyers and nonbuyers, in contrast to the usual belief among marketers that higher income groups and women are more likely to buy green.
Of the many consumer segmentation schemes available, PRIZM has emerged as particularly useful for identifying green energy buyers. The 2001/2002 survey identified five PRIZM clusters as containing those most likely to participate in a green pricing program (see Tables).

PRIZM cluster green indices

Cluster name	Green index*
Urban Achievers	4.5
Blue Blood Estates	3.5
Winner's Circle	2.5
New Empty Nests	2.4
Pools and Patios	2.2

*Green index is the normalized ratio of participants to nonparticipants in that segment, and is thus a measure of the relative likelihood of a member of that segment buying green.

PRIZM cluster definitions

Urban Achievers	Mid-level white-collar urban couples. Often found near urban public universities, these neighborhoods are ethnically diverse with a blend of youth and age. Single students mix easily with older professionals who work in usiness, finance, and public service. Affluence is "middle" among the clusters.	Age groups: 25–44, 65 and older.
Blue Blood Estates	Elite, super-rich families. Established executives, professionals, and "old money" heirs live in America's wealthiest suburbs. They are accustomed to privilege and live luxuriously. One-tenth of this cluster are multi-millionaires.	Age 45–64.
Winner's Circle	Executive suburban families. These "new money" families live in expensive mini-mansions in major metropolitan suburbs. They are well-educated executives and professionals who are married with teenagers. Big producers and big spenders. Winner's Circle families enjoy globetrotting.	Age 45–64.
New Empty Nests	Upscale suburban fringe couples. Hard work in professions and industries has rewarded New Empty Nesters with the affluence that comes from double incomes. Most of these married couples are in their "postchild" years, are far more conservative than Young Influentials (a cluster of high-tech managers and professionals), and live in the Northeastern and Northwestern U.S. Affluence is upper middle, ranked 15 among the clusters.	Age 45 to post-65.
Pools and Patios	Established empty nesters. Empty-nester executive and professional couples are living the good life in their "post-child years." Their dual incomes support rich active lives filled with travel, leisure activities, and entertainment. Many live in the densely populated Northeast corridor of the U.S. Affluent, in top 10 among clusters.	Age 45 to post-65.

Although these market segmentation schemes can seem rather imprecise and ad hoc, there's more value here than one might think. Most U.S. households are already classified into PRIZM clusters based on zip codes. Most marketing, from direct mail to television advertising, already makes use of PRIZM-like market segmentation techniques. It's only the energy industry, with its legacy as a regulated monopoly and an attendant indifference to consumer preferences, which has been slow to recognize the value of such methods. Greater use of these techniques will clearly lower marketing costs and increase green power market penetration in the future.

What's the implications of this for your market research? First, that some market segmentation work has already been done, and that you may want to just use the PRIZM or another similar method, rather than coming up with your own. Second, that commonly held ideas (such as gender and income as strong correlated with green energy purchasing) are often wrong, and market research needs to be designed carefully so as to exclude untested assumptions.

The market research process

Successful market research requires planning and organization. We've put together a flow chart that lays out the key steps and decisions in the market research process.

As the flow chart shows, market research is a complex process. Many utilities choose to outsource some or all of it. But even if you do so, it's worthwhile spending some time understanding the concepts and execution of market research. To that end, here are some recommended books on the topic:

G. A. Churchill Jr., Basic Marketing Research, 1988, The Dryden Press
D. A. Dillman, Mail and Telephone Surveys: The Total Design Method, 1978, Wiley & Sons, Inc.
T. L. Greenbaum, The Practical Handbook and Guide to Focus Group Research, 1988, D.C. Heath and Company
G. L. Urban & J. R. Hauser, Design and Marketing of New Products, 1993, Prentice-Hall, Inc.
S. Sudman & B. Wansink, Consumer Panels (2nd Ed), 2002, American Marketing Association
G. A. Churchill Jr., Marketing Research: Methodological Foundations (5th Ed), 1991, The Dryden Press

Working with market research firms

One important decision is whether you want to outsource some or all of your market research. Market research is specialized knowledge, and it often makes sense to work with a skilled vendor of market research services. Most utilities outsource the grunt work – that is, the actual data collection (e.g., surveying). Some outsource other components as well, such as preparation of the survey instrument or data analysis. Because outsourcing is so common, we provide detailed instructions on how to select a market research vendor, and how to evaluate their work.

What to ask for in the RFP

A utility soliciting market research vendors should always get at least 3 bids on any project to assure a competitive price. For a list of market research vendors, the American Marketing Association Website can provide the names of several good companies.

The vendor should respond to the client's Request for Proposal with a proposal that is complete, on point, on time, professionally done, and cost competitive.
The vendor should provide examples of previous reports, for an evaluation of the level and quality of their analysis. [When the client is shown examples of previous work, that client's name should not be apparent on the report. This is an indication the vendor is likely not to respect the privacy of its clients.]
The vendor should provide 3-5 relevant references.
The vendor should provide the client with a project outline, including a timeline with key milestones; information regarding the staff responsible for the study and their relevant experience; and a breakdown sheet of costs by task and an average cost for each survey method used, as well as options for task and survey methods at alternative costs levels.

What to look for in vendor staff and experience

The vendor should have experience in:
- conducting market research studies with the customer group the client is interested in surveying;
- developing questionnaires and conducting analyses on issues related to the client's industry;
- standard data collection methodologies - phone, mail, fax, and online, and mixed methods such as phone-mail-phone – and should be able to make recommendations for when to use each approach.
The company should be established and active in the market research field for over 5 years.
The client should be provided with a single point of contact – a project manager with a manageable workload (the rule of thumb is no more than 3-4 projects) and at least five years experience in market research.
The vendor should have a statistician and energy industry experts on staff.

What to look for in a vendor's standards and practices

It is best if a vendor manages their data collection activities in-house. This promotes better quality control and communication between the project team. If they do sub-contract to other data collection firms, then it is important that they have had a long-standing partnership with the third party.
The vendor should subscribe to the market research code of ethics, as described by such organizations as CASRO (Council of American Survey Research Organizations).
The vendor should have stringent quality assurance standards in place for conducting interviews, verifying completed surveys, coding open-ended questions, etc. If desired, the client should be able to monitor telephone interviews remotely.
The vendor should be willing to sign a contract or statement of work that includes final estimates of costs, timeline and deliverables.

What the vendor should do before the study: Preparation

Study objectives should be identified up front with the project team and communicated clearly to this team. A written agreement that includes the project objectives, and approved by all principal internal and external clients, is a good idea to avoiding misunderstandings later.
The vendor should provide the client with a project timeline including key milestones;
Before data collection can begin, the survey sample must be properly identified along these dimensions:
- Sample Frame: to identify the relevant population, sample groups, and the criteria for classifying respondents into groups.
- Randomness: to ensure there is a proper representation of the overall population, samples should be obtained in a random fashion, enabling everyone an equal chance of being selected.
- Size: samples need to be of statistically significant size. Sample sizes of 400 are utilized most often among market researchers. Assuming the sample has been randomly selected, results based upon 400 completed interviews have a error range of +/- 5 percentage points with a 95 percent confidence level. When a sample is stratified in advance, it is important that each group is large enough to conduct individual analysis and meaningful comparisons. Therefore, sub-groups, or quota groups, should ideally have at least 100 in each mutually exclusive cell.
The client should receive a draft version of the survey instrument for comment and input.
The survey instrument(s) should be designed to accurately measure or test the research issue in question. Measurement scales need to be consistent and appropriate for the issue under investigation. If data will be combined from more than one source, there needs to be consistency in sample and time frames and survey instrument measurement scales.
An analysis plan should be drafted prior to survey design and collection. This plan will detail the analyses the vendor intends to conduct and may include hypotheses and anticipated relationships in the data.

What the vendor should do during the study

The vendor should pretest the survey – with the client's participation, if desired - prior to launching it.
The vendor should provide daily or weekly status updates on the project regarding completion rates or problem groups.
If it is a telephone survey, the client should be able to monitor interviews while the study is being conducted.
If the study was conducted via the mail or over the Internet (where there are typically lower response rates) the vendor should consider non-respondent bias and determine whether or not non-responders need to be evaluated to assure the integrity of the data.
If the sample was stratified into sub-groups before data collection, weight factors need to be calculated and applied before the data is rolled together for final reporting.
During data collection respondents should be:
- treated with respect and thanked for their time;
- offered incentives when appropriate;
- offered a mechanism to document complaints and/or requests to be taken off the research "list" for future studies;
- guaranteed confidentiality of individual responses when requested.

What the vendor should do after the study: Deliverables

The client should help determine the specifications for data tables and the outline of the report.
The client should receive a draft version of the report for comment and input before distribution.
Included in the final set of deliverables, the client should receive:
- the raw datasets which have been cleaned, a data map or codebook and the data tables/tabs;
- a final report which includes key findings, recommendations, detailed analyses and any findings based upon statistical differences in the data.
- a formal presentation of results, either over the phone or in-person.
Results of the market research should be defendable and make sense to those who deal with the issues being investigated. Results need to be incorporated with real world data and information to be most effective.
The project should come in on time and on budget. The vendor should not require more money for the project at any mid-point in the study, unless there have been major changes in project scope or in estimated response rates or incidence levels.

Model survey forms

Market research can take many forms: focus groups, deliberative polling, statistical analyses of buyers and nonbuyers, and so on. But its most common form is the survey. Here we provide some sample survey forms that utilities can use as guides in writing their own surveys.

For those interested in evaluating their green pricing programs, we've included survey forms designed for both program participants and nonparticipants. Surveying both groups allows you to better understand how these two groups differ, which can help you better target your marketing.

A short survey that measures overall attitudes and beliefs about energy and renewable can be found in the back of a June 1999 NREL report entitled, "Colorado Homeowner Preferences on Energy and Environmental Policy."

If you're specifically interested in customers' views on residential grid-tied photovoltaic systems, see Appendix J in the NREL report, "A Market Assessment of Residential Grid-Tied PV Systems in Colorado."

Deliberative Polling

Nebraska Public Power District recently conducted a first-ever deliberative polling effort durring the summer of 2003. The results received indicate a high level of support and interest in alternative energy sources such as wind and methane, as well as strong approval for the polling process. After collecting initial opinions via a telephone survey on energy alternatives in June, poll participants were invited to attend a daylong customer meeting in August to read, think, discuss and ask questions of experts and advocates of energy generation alternatives and share their opinion. Developed by a professor at Stanford University, Deliberative Polling measures participants' opinions before and after they are educated about a specific topic. "I think we learned as much from the process as our customers did about energy alternatives," said Frank Thompson, NPPD Renewable Energy Development Manager. "The results indicate overwhelming support for wind power and a great deal of interest in other renewable opportunities, such as methane." After eight hours of discussion at a customer meeting held in Grand Island on Aug. 9, 2003, the customers gave NPPD high marks for the process. Ninety-three percent considered the experience very, if not extremely, valuable. Specifically, the polling results show that 96 percent think 200 megawatts of wind power should be added to NPPD's diverse generation fuel mix and 37 percent think NPPD should add more than 200.

Customers voiced strong support for the methane (animal manure) projects described during the day. Eighty one percent think NPPD should pursue adding five MW of power from methane gas to its generation mix. Using Nebraska-based resources was important to the participants, and 94 percent said all customers, not just those bill-paying customers that want power produced from renewable resources, should pay for the costs of renewable energy costs. NPPD's use of the deliberative polling process is significant because it is the first new deliberative polling data on energy alternatives in five years, is the first application by a public power entity (customer-owned versus investor-owned), is the first Deliberative Poll in the Midwest and reflects a rural sample of data (78 percent of the telephone survey participants reported they lived in an area of less than 50,000 people).

Other observations collected from the deliberative polling process included customer satisfaction in the service provided by NPPD and other retail utility suppliers in the state. In addition, both coal and nuclear power options fared better than they have in other areas of the United States in similar polls conducted five years ago. "Our customers took the time to share their opinions with us, and that's an important part of the public power process in Nebraska," said NPPD President and CEO Bill Fehrman. "We appreciate their input and will consider it as we move forward in our resource planning efforts." The Deliberative Polling effort was primarily funded by grants from the Western Area Power Administration (U.S. DOE Wind Powering America Funds) and the Nebraska Energy Office. You can obtain a detailed copy the the polling result by clicking on the following link:

NPPD Deliberative Polling Final Report