Chapter 6 Doing Your Homework: Evaluating Your Ideas Product Management


  • Checking out market research from a product management perspective
  • Completing competitive intelligence
  • Validating your ideas with customers
  • Taking the first steps in financial forecasting

For every idea that makes it to market, many others aren’t as worthy of pursuit as the one or two that you finally focus on. This chapter covers market research as it applies to
product managers. For example, understanding what your competitors are doing — and
not doing as well as they could — is important research as you find an opening for your
product to be successful. Validating ideas is critical to avoiding the many possible pitfalls of customers not accepting your product. And some simple calculations can help you sell your idea into the company by showing that the product can be profitable. In fact, you’ll reuse these tools and techniques throughout your product’s life cycle as you need to make further validated decisions about product next steps.

Understanding the Importance of
Market Research and Competitive

Market research is the way in which product managers gather information about customer needs and market drivers. If you want to gather information from actual customers to make a decision, then you need to understand and use market research. Competitive intelligence is a subset of market research. When you investigate your competition, you use market research techniques and concepts to understand what your competition is doing today and gain insight into its plans for tomorrow .
Market research helps avoid the four-walls problem that stems from using only the
collective wisdom of the people in the meeting room to make a decision. The solution is to step outside the confines of the company to get another vantage point.

Here are some good opportunities to gather information and create solutions:

  • Identifying market needs and customer problems
  • Prioritizing possible features
  • Deciding on new target markets
  • Segmenting a market
  • Determining your market share
  • Measuring customer satisfaction
  • Learning about the competition
  • Deciding on final pricing
  • Defining or testing new product concepts
  • Creating and testing messaging

The value of market research is almost infinite. And the reason that you should actively
seek information is because in every decision you make there is an implied hypothesis. As you may know from science class, a hypothesis needs to be tested. In the terminology of product management, an idea requires validation.

The investigative loop shown in Figure 6-1 is a common concept used in product
management. Usually you start with a discovery; you then form a hypothesis, which is a set of assumptions about your product, and then validate and test it. From there you
experience some learning and apply it to adjust your plans.

FIGURE 6-1: Investigative loop.

Chapter 3 discusses different phases of the product life cycle. T o make decisions in every one of those phases, plan on gaining real market insight and information to make the most informed decision that you can.

Subdividing kinds of market research

In product management, the core of any solution begins with an in-depth understanding of the needs and problems that your customers are facing that you think you can solve with a product offering from your company . Initial understandings of any problem can be very vague; in fact, if you’re emphatic early on that you know exactly what the problem is, you should look again.

Qualitative and quantitative market research

Because the problem definition is so hard to create and is critical to success, the best way to get a handle on all aspects of the problem is to talk to people who are experiencing it. Yup. That’s it. Talk to them. Ask questions about how they do things now and what their biggest challenges are and listen. Focus at least initially on a conversation as opposed to a questionnaire or survey . Chapter 5 discusses how to hold these conversations with your customers.
In market research terms, the conversation is called gathering qualitative data. Once you have gathered qualitative information about your customer’s problem, you can use
quantitative data, such as a survey , to gather more concrete information.

Primary and secondary research

Another consideration when conducting research is whether you’re able to buy research
done by specialist companies, use research provided by the government, or create your
own research study . Research that you pay for and therefore own is called primary
research; it’s known only to you. Research that you don’t do yourself or purchase is called secondary research. Check out T able 6-1 for examples of secondary and primary research sources.

TABLE 6-1 List of Research Data Sources

Secondary Research SourcesPrimary Research Sources
Government sources of demographics and
economic statistics; trade associations
Customer interviews
Specialist reports: The Economist, Harvard
Business Review , Forbes Magazine, Articles
about the industry or competitors
Customer relationship management (CRM) data, including the
most common requests of customers that sales receives and the problems that support captures
Websites of competitors; annual reports; press
Support database
Industry analysts and research organizationsCustomer surveys and focus groups
Internet searches for industry and market data
and trends
Research done in other parts of your company that may apply

Looking for the right place to start

When starting in on a market research project, know that you’ll be reading a lot of material both quantitative and qualitative in nature in printed material and on the web just to get a few nuggets that are of real interest. Think of it as panning for gold. Finding a few flakes of valuable gold is a cold and wet business, but without the flakes, you can’t continue your work, and no one will join you on your journey to find even more gold.

Qualitative secondary research

The typical starting point in conducting research is to read any and all secondary research that you can get your hands on. Y ou’re looking to understand the lay of the land. What do people who experience this problem understand the issues to be? Where do they believe upcoming technology and business solutions are coming from? Larger companies often subscribe to secondary research from specialist industry analysts. If you’re short on market research funds, search the Internet and look to sites that share presentations, such as slideshare.

Quantitative secondary research

Fantastic! You’ve identified a promising market or problem to solve. Do enough people
have this problem? Is this problem growing or shrinking? Using secondary research often starts with qualitative analysis and then follows it up with quantitative market sizing numbers of all kinds:

  • How many companies or customers have this problem?
  • Where are they located and what are their demographics?
  • What common characteristics do they have (company size, revenue, business profiles,industry)?
  • Are any companies currently solving the problem you have identified. If so, which ones?
  • Is this market growing or shrinking?

T o find quantitative secondary research use the list in T able 6-1 to locate any research
sources that might be available. There may be some available from industry analyst firms such as Gartner , IDC, or Forrester . These reports are often somewhat costly , but they can contain very useful data. Check with your market research and competitive analysis department (if you have one) to see what they might have on hand or have access to. Sometimes they pay for subscriptions to the analyst reports. You may find after an extensive search that there is nothing available, but at least you know that no one else who might be investigating the opportunity has any more data than you do.

Qualitative primary research

To perform qualitative primary research, you’ll need to talk to customers. You may not be able to visit each customer in person, but your focus is to get answers to the questions that you can’t find in publicly available (also known as secondary) research. By conducting in-person interviews, focus groups (which require a lot of expertise to do right — you’ll want to hire an expert to do this for you), or customer council meetings, you can ask more specific questions about your hypothesis and get a deeper level of understanding about the customer problem you’re trying to solve.

When you believe that you’ve identified your target customer , you’re ready to develop a
research plan including the list of questions (such as the following) that you want to ask
the target in a conversation. Review Chapter 5 to determine which persona types you
should target your questions to.

  • Can you tell me a bit about yourself? What is your job title? What education did it take to get this position? What is your age range?
  • Do you experience the following problem(s)? Which other problems do you also experience?
  • If you do, how do you overcome it?
  • Can you show me the steps you take to overcome it?
  • Do you have any ideas or suggestions for a product that might help you with this?

Undertaking the Market Research

Doing more formal market research right means going through a series of steps designed to get you the information you need. The following sections help you do just that, leading you through the kinds of questions you should ask and the research methods product managers everywhere use.

Spelling out the market research process

Whenever you have a question and decide that conducting market research is the right
solution, keep in mind that following an established process will lead you to better results:

  1. Decide on your research objective.
    What are you trying to achieve with the research — assess whether the
    company should develop a specific product line? Change the direction of an
    existing product in development? Simply validate that a new feature will meet
    customer requirements? Y our objectives also determine the scope and cost of
    the investigation. Deciding on whether to enter a $300 million market
    opportunity justifies more effort and spending than prioritizing a few features
    for a $99 product.
  2. Determine what questions you need to answer to satisfy your research

    Write down an initial set of questions and discuss them with your colleagues
    and peers to get input. They may point out that the questions are very broad,
    or they are too specific and won’t allow for you to potentially uncover any
    additional valuable information.
    Here’s an example:
    Too broad: What kinds of exercise equipment do you use to stay healthy?
    Too narrow: Do you use legs weights to stay healthy?
  3. Choose the best research technique(s).
    As a rule of thumb, you move from secondary qualitative research through to
    primary quantitative research as shown in Figure 6-2 . In reality , you may go
    through one of these categories pretty quickly and then spend a lot of time
    getting your primary qualitative research right. (See T able 6-2 later in the
    chapter for a list of the most commonly used market research methods.)
  4. Design your research study .
    Designing a good research study requires a lot of expertise. Use the options in
    T able 6-2 to determine what type of research you need to conduct. If possible
    work with someone who has done it before, such as a peer in your company ,
    your internal market research department, or an outside vendor . It is easy to
    accidentally design a study that has a lot of bias, so be careful to scrutinize
    your design accordingly . If you can’t find anyone to help you design your study
    or to critique it for you and provide feedback, we recommend you find a book
    on the topic that covers it in an in-depth fashion.

Remember Asking the right questions is a critical part of getting good answers. The
following section has more specifics on this topic.

  1. Conduct research.
    Conducting research may take you as far as across the world or as close as a
    car ride to a nearby town. It is highly dependent on what type of research you
    have decided to conduct. Customer interviews may be held in person or on the
    phone. Focus groups may be held in multiple cities or conducted virtually .
    Customer councils often involve having customers fly in to your location.
  2. Collect and analyze data.
    Qualitative data is basically listening to people answer your questions with few
    boundaries. Decide how you want to record this information. In person, avoid
    taking notes on a computer; the presence of a screen creates a barrier between
    you and the customer . If the customer agrees, recording the conversation is
    always an option. A good technique for taking notes while recording is to make
    a note of the recording time when the customer says something interesting.
  3. Present findings to take advantage of new knowledge.
    Going through all the trouble of getting data to support a point of view isn’t
    going to do you any good if other parts of your organization don’t understand
    it. Use the tips in Chapter 17 to make sure your message is heard. A
    convincing presentation and/or written report should be short with lots of clear
    graphics. For presentations aim for 10 slides and no more than 20. Have
    someone with good graphic skills review your materials, and you should
    practice before any presentation.

TABLE 6-2 Market Research Methods

Type of Market
DetailsWhen to UseRelative
EthnographicAnthropological-based observation and
questioning. Usually conducted one-on-one or in small groups.
Great for the conceive stage when the concept is still unformed and a lot of
unknowns exist.
Customer panelsOngoing dialogue with a static group of key
customers who provide real-time market
intelligence. They’re an early alert system to
identify upcoming opportunities.
Can be gathered in a series
of meetings and fed into
early product development
or , more typically , ongoing
product evolution.
Usability testingEvaluation of customers using a product and
providing input. Can be done at the customer site,
in a lab, or virtually . Services exist to conduct this research for you.
During development and the
testing and validation phase
of product development.
Qualitative input from a few (3–20) people. It
works well when defining positioning of key
Any time in product
development. More
commonly used earlier in the
product development cycle.
Focus groupsSmall group discussions facilitated by third-party specialists on a focused topic such as obtaining rich qualitative data; issues that exist with the product as currently developed; or refining
positioning, names, and packaging.
Primarily used for fine
tuning a product and in the
market launch stages.
Quantitative information to support or reject a hypothesis. Also can be used to get qualitative
input from a small group of people.
Interviews: At any time in
the product process.
Surveys: Primarily in full
production and market launch stages.
Web or print
Quantitative input to fine tune and prioritize possible product options. Beware that how you ask
questions can tilt the responses.
Primarily in full production
or market launch stages.
Low to
FIGURE 6-2: Sequence of market research.

Asking the right questions

Asking questions. It seems like such a simple thing to do, and then you ask a question and get a response that isn’t at all what you asked for . Now imagine trying to get answers to specific questions that will guide product development. Yes, it’s hard.
Here are some things to keep in mind as you formulate your questions:

  • Consider how specific you should be. A good gauge is knowing how specific you need the answer to be. At the beginning of the process, the questions are generally qualitative and very broad. Open-ended questions that start with why , what, how , where, and when are a great way to go. Then shift to quantitative questions. Quantitative questions have more specifics than qualitative questions, as you are seeking to get hard data to use.
  • Ask how important something is, not just whether a customer desires it. If you ask a customer whether she wants a feature, the answer is almost always yes. For example, if you’re evaluating two specific features, you may want to ask the following questions and place the responses in a table such as the one in Figure 6-3 . On a scale of one to five, how much do you like features one and two? On a scale of one to five, how would you rate feature one’s and feature two’s importance? In this case, you’d develop feature two before feature one because 15 is greater than 5.
  • Ask follow-up questions. In qualitative research, continue the conversation that the customer has started. The topics raised may not be on your list of questions, but this is the time to find every nook and cranny of opportunity
  • Avoid bias. If you ask questions with enthusiasm for your preferences, you are more likely to get biased answers. If you can’t avoid asking a question with bias, get someone else to ask the questions while you take notes with a poker face. This applies to both the question content as well as the inflection with which it is delivered.
FIGURE 6-3: Charting customer answers.

No matter how great a product manager you are, always have someone look over
your questions. What seems crystal clear to you often won’t always make sense to
someone else.

Examining market research methods

Table 6-2 covers most of the popular methods of market research. When you start a larger project, work with a market research specialist, either inside or outside your company , who can advise you on which type of research method will work best to achieve your research objective. If you contact an outside organization, make sure that it doesn’t specialize in only one type of market research. Those companies will inevitably want you to use only their method.

Figure 6-4 gives an overview of market research methods. On the left are high-context
methods, meaning that you need to conduct them in person and often onsite. High-context methods give you a broader understanding of the world that your potential customer lives and works in. Y ou use these methods mostly in early stages of decisions to provide a background to further research. On the right are low-context methods , which are very arm’s length and impersonal without much background information. Low-context methods are great for getting quantitative results — the numbers.

FIGURE 6-4: Overview of market research methods.

Studying Competitive Intelligence

Competitive intelligence is also known as competitive analysis. It’s the intelligence
acquired about your competition that allows you to compete more effectively . Competitive intelligence uses a lot of the techniques from market research. Check out the techniques in Chapter 5 .

Identifying competitors

Here are some common ways to discover your competition:

  • Reading product reviews and comparisons gives you an idea of who your customers and industry pundits believe your competition to be.
  • Y our own salespeople are a great early warning on newcomers or changes in what your existing competitors offer . Offer a (small) reward when sales comes up with a new competitor or feature that you should know about.
  • Industry analysts create periodic reports that compare your company to other companies.

Remember In stable markets, focus only on the top two or three competitors. If you can beat these competitors, you should be able to easily beat the smaller ones. In emerging
markets where what customers really value is less clear , you want to keep track of a
few more competitors until the customer value picture becomes clearer .

Collecting all the competitive intelligence possible

For a true 360-degree view of your competition, your research will encompass the areas in the following sections.
All this information creates a map of potential competitive moves. You need to synthesize the key indicators. If you find this synthesis challenging, explain what you’ve found to a few colleagues to uncover the likely next moves of your competition. Present any significant issues to your manager and decide what the next steps will be.


Consider these bigger-picture issues:

  • External performance: Are your competitor’s sales increasing or decreasing? How about their profits? How many people work there, and what is the revenue per person?The trick is to look for changes in numbers from last year to this year — and even the years before that. If the company you’re tracking isn’t public, search press releases for whatever information you can find. Check the company’s annual report if it’s public,funding announcements if it’s private, and press releases (all available online).
  • Funding and spending: Are competitors spending more or less money on sales,marketing, and research than they did before? Y ou can compare their spending to your company spending. Beware that your competitor’s company could be structured differently from yours, so investigate large differences between companies. For private companies, you can sometimes find more data on the website of the venture capitalists about this. For public companies, the notes in the annual report as well as some analysis of the profit and loss statements can give you clues (particularly if a company breaks its profits and expenses down for each type of business it’s in).
  • Management: What is the range of skills found in competitors’ management? Do the managers lack certain expertise? How diverse is their membership? How well are they connected to key influencers in their industry? Y ou can often find this information on the “ About the company management” page on their website, in the LinkedIn profiles of their executives, and in interviews that may have conducted with their management team by magazines and online news sources.
  • Attitude: Are competitors aggressive? Complacent? Do they like risk or profoundly avoid risk? Which quadrant are they in on the performance and organizational health scale shown in Figure 6-5 ? Gathering this information can be done by having conversations with your salespeople about what they have seen the competitor do, from former employees of the competitor and from articles written about it.
FIGURE 6-5: Performance and organizational health quadrant.

Product comparison

Most competitive analysis that product managers do is typically a side-by-side
product comparison chart that compares only features. Remember to add in aspects
of service, warranties, finance, service, and reliability that may add more strength to
the competitive argument. Figure 6-6 is a great way to organize your thoughts in a
way that educates your customers and internal audiences at the same time.

FIGURE 6-6: Product comparison table.

Marketing and distribution

Marketing and distribution are other important areas of competitive analysis. Think about the following:

  • Marketing: What does your competitor’s brand stand for? How well does the market believe in that positioning? What is the competitor’s positioning at the product level?What are it’s marketing programs? Which trade shows does it attend or avoid? How sophisticated are its digital marketing efforts? Which keywords (terms that, when typed into Google, result in ads related to them) does it target for AdWords and online advertising? Search for some likely terms and see if their ads show up. Is the company’s marketing more sales, product, or technology focused?
  • Distribution channels: What are the competitor’s distribution channels? Does it go direct, use distributors, or have special relationships with certain channel partners? Most importantly , what advantage or disadvantage its distribution model provide?
  • Market share: What is your market share? What is your competitors’ share? In mature markets, the market leader usually has at least twice the market share as the next biggest competitor . Market share can lead to market dominance and possibly complacency .

Porter ’s five forces

In 1979, Michael Porter wrote “How Competitive Forces Shape Strategy” in the Harvard
Business Review . His concepts, shown in Figure 6-7 , are as valid today as they’ve been
throughout history . Consider each of these aspects when evaluating your competitors and your market overall. T o use Porter’s five forces, answer the following questions for your product or market:

  • What is the bargaining power of your suppliers when they negotiate with you?
  • What is the bargaining power of your buyers when they negotiate with you?The fewer or more specialized your suppliers and buyers, the more power they have in controlling negotiations and determining pricing that is less favorable to you.
  • What about potential new companies and services coming into your market? Can these entrants compete easily , or is it difficult for them to compete with you?
  • What about substitute products? What kind of products could replace your products in solving a customer need?
  • How strong is rivalry in your industry? Are your competitors aggressive in taking business from you?
FIGURE 6-7: Porter’s five forces.

A couple of handy acronyms

PESTEL and SWOT . No, we haven’t forgotten to use spell-check. These concepts are two
pieces of the competitive analysis puzzle.
PESTEL stands for political, economic, social, technological, environmental, and legal.
PESTEL analysis is used to understand the overall context within which you do business
and within which your product is evaluated. To complete a PESTEL analysis, answer the
following questions. What is the overall social and business environment that you operate in? What is changing? What are key trends in each of these areas? How these stresses impact your competition isn’t always the same as the way they impact you. List each factor and examine your situation closely to see where opportunities and threats lie.
SWOT is an acronym for strengths, weaknesses, opportunities, and threats. The first two
terms, strengths and weaknesses, refer to the state of the competitive company that you
are conducting a SWOT analysis for . The second two terms, opportunities and threats,
refer to external impacts on your competition, their markets, and their products. By doing a SWOT analysis for each main competitor you can get a more accurate view of who you are actually competing with and where they are strong and weak. This forms the basis of your own strategy for beating them in the marketplace. Use Figure 6-8 to understand how the different information in each quadrant interacts to give you a synthesis of

  • Strategies the company can exploit to take advantage of opportunities
  • Strategies the company must deal with to take advantage of opportunities
  • Strategies the company can use to defend itself from threats
  • Strategies the company must deal with to defend itself from threats
FIGURE 6-8: SWOT analysis.

Keeping track of the competition

Competitors aren’t standing still. T o make sure you keep on top of their moves, allocate
regular time to competitive evaluation — monthly or quarterly , depending on the industry .
Consider these suggestions:

  • Create a weekly Google Alert for whenever the competition is mentioned (search Google for the term “Google Alert” to learn how to create alerts). For example, if your company sells smoke detectors, create an alert for the phrase smoke detector . Then every time this phrase is mentioned in the news or on a website, Google will send you an alert letting you know where and when it appeared.
  • Check your competitor’s website monthly for any announcements or updates to its product. This time frame should be enough that you pick up significant changes. Make written notes about your findings that you can refer to later if needed.
  • For publicly traded companies, listen in on the analyst call quarterly . Download and read through the latest financial report. What has changed? What issues has the company listed as risks?
  • Call channel partners monthly or quarterly for a chat on how business is going, whether you’ve lost or gained competitive ground, and, if so, to whom and why
  • Plan a full-blown annual review to present to your management. Preparation for the review will force you to pick up any details that you’ve missed along the way with your weekly , monthly , and quarterly work.

One fantastic benefit of all this tracking work is that doing it in small pieces takes
less effort than having to whip it up suddenly on an annual basis. And your
organization will have full confidence in your thinking in other areas if you’re current
on your competitive analysis.

Reality-Checking Y our Ideas and

Chapter 4 provides some tools and ideas for coming up with new product ideas, and the
market research and competitive analysis section of this chapter shows techniques for
exploring potential markets further . One other technique that you may want to use for a
market or product that is brand new is doing some additional reality checking.
For newer markets, finding a really good winning idea often means direct validation with
actual potential customers. The goal in doing this is to present enough of an impression of a product (whether it is a description, data sheet, or working demo) that customers can imagine themselves buying it and then tell you what they think.

Using a simple validation process

You have a product idea that you think may work for certain customers. That’s wonderful.
You now have two choices:

  • Spend a lot of money developing it before you talk to a customer at all.
  • Spend virtually nothing by creating a product concept and then asking target customers what they think about it.

Remember that you may want to test out quite a few potential products and the associated hypothesis. The wise course of action is to take the second option and see what customers say first.
Here’s a simple and relatively fast validation process:

  1. Create a hypothesis about a potential product.
    For example, “Customers who use digital forms want to fill them out online and
    have them automatically routed from one person on to another for signature.”
  2. Create a list of target customers.
    In this example, the target customers might be multinational companies that
    sign forms across the country and possibly across the world; small digital
    media houses that need sign off on final artwork for various projects; and small
    companies that need forms signed as part of their workflow inside and outside
    the company .
  3. Create certain artifacts to prove to customers that this product is real.
    The artifacts are a datasheet, a price list, a short presentation on the features
    and benefits of the product, and a demo done in Microsoft PowerPoint or Word.
    Note that you create no actual product, but you create enough of the product
    concept to present it to customers so they get a realistic idea of what would be
  4. Hold individual meetings with several people from each target
    customer group and see what they say .

    Is the target group interested in the product? Which features does each group
    deem critical? How are they solving the problem today?
    When you have a list of each of the target groups that is interested in the product, you can
    go back and research how big that target market is and then create a rough, back-of -the-envelope estimate of how much potential revenue and profit you could generate. With actual customer feedback, you can proceed to more in-depth planning with more confidence.

An example of product validation

Zappos, the online shoe distribution company , tested its concept by making arrangements with actual shoe stores. Rather than building out a warehouse and distribution system first, Zappos tested the concept by putting pictures and prices of shoes online and then filled orders from a physical shoe store. In this way , it avoided upfront costs of a warehouse until it had validated its concept.

Crunching the Numbers with Financial

At some point, the decision about which, if any , products to move forward with will come down to profitability . And at this stage of product development (or non-product
development), you don’t want to get bogged down in enormous reams of financial data and overly complex forecasts. However , you really do need to support your hypothesis with some simple financial analysis and a list of assumptions that went into creating them. To do this, you want to create a draft profit and loss statement, do a break-even analysis, and calculate the return on investment (ROI). Chapter 9 has more detail about how to create these financials.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: