Monday, April 21, 2014

How to get top quality market research?

by Mathieu Ollier

Nowadays, every OEM has a ‘Market Intelligence’ division to gather and analyze data. But, obtaining “intelligent” data about the market – data that isn’t flawed, sloppy, or incomprehensible – isn’t easy.

Let’s start at the beginning. The role of the Market Intelligence division is usually to gather and analyze data such as sales, market share, spare parts prices, terms and conditions, etc. Obviously, access to sound, high quality data is paramount. While most of us have relationships with trusted and reliable research partners in familiar countries, eventually we have to carry out market research in places that are more ‘exotic’, and where concepts like ‘quality’ and ‘time’ are (much) more fluid than we would like them to be.

Since many of our projects require gathering data from outside sources, we have experience dealing with market researchers around the globe. We have had to deal with the whole spectrum of researchers, from ‘great’ to ‘awful’. In this article, we’ll focus on the ‘awful’ and discuss how to avoid massive train wrecks and get top quality market research.

Simply put, market researchers are usually better at executing than they are at thinking. That means you must do most of the thinking before the research begins. Shifting course while moving at full speed risks that train wreck. Think of a kindergartner crossing the street; you wouldn’t let him do it without supervision. No, you tell the child what to do, then you take his hand, look left, then right, then left again, and walk carefully, following the crosswalk markings. Now, how does this metaphor translate to the real world? Let’s see:
  • Research is a commodity. Few differences separate two market research agencies doing the same work. Granted, a company may have specific capabilities, but they are usually upstream (e.g. cross-referencing) or downstream (e.g. data validation), and not pertinent to the research itself. So, if you receive price offers for your research that are extremely different, think carefully. It means that one of the companies is not going to do what you want (a different methodology), or that it is cutting corners to appear attractive, or that it has never done this type of work before and does not have enough experience to determine the actual cost, or maybe that it plans on doing something illegal like not paying taxes... None of these scenarios is attractive; together they mean that someone may be taking you for a ride.
  • Market research companies have a different definition of ‘high quality’. As we have mentioned before, some companies focus on speedy execution, instead of looking up from the steering wheel in order to avoid oncoming traffic. Consequently, you can’t be suspicious enough. Until you know that a company is reliable, assume that everything they do is wrong and double-check all that you receive from them.
  • Do not let the researcher decide the format of the data they will return to you. While you know exactly how you want to use the research output, the researcher does not. He has no idea and no real incentive to put it into a format that will be easy-to-use and convenient for you. Make sure to provide a template, and to brief your research partner on how to use it. Ideally, you would do this during a formal kick-off, along with the explanation of your expectations, a reminder of the research timing, as well as the expected methodology. If you use different providers for different countries, chances are that the same team within your organization will use the data. Having the data in the same format will make their lives a lot easier. They’ll be able to focus on value-added tasks like analysis, not reformatting long Excel files.
  • Plan a soft launch rather than a full-steam-ahead launch. Nothing should prevent the researcher from carrying out a soft launch, (that is, research for only a small sample of the agreed scope), and provide you with that data. Look at it, check that it makes sense, and that there are no major flaws in the process you defined. If the sample looks good, you can give your researcher the go-ahead to get the rest of the data.
  • Ask for data extracts / work-in-progress files. Researchers are notoriously late. I always suspect that they don’t start when they say they do, but procrastinate for three weeks, like a high school student with a term paper. (I know: I have trust issues myself ... working on it). Asking for scheduled research updates and work-in-progress files will reduce your stress level. If they can’t provide the update to you three times in a row? Something is very wrong. Time to put on the brakes and figure out what’s happening.
  • There is no such thing as ‘we’ll do it later’. If a piece of the data is missing or is recorded incorrectly, the research agency may promise to fix it later. Don’t buy that; it most likely won’t. And when all is said and done, and nobody remembers why that data was not included in the research, guess who’ll look bad? Instead, insist on immediate corrections.
  • Data validation is a key step of the process. Researchers usually assume that they’re off the hook the moment they give you the data. Wrong! Until you are confident in the quality of what you received, the researchers should be responsible for it. Ideally, data validation should also happen on their end, but experience tells us that their “data validation” is not very thorough. There are always wrinkles (e.g. missing zeroes, typos, referencing errors, etc.) in the data you first receive. Make sure to have an explicit, written agreement about data validation responsibility, what validations rules will be applied to check its quality, as well as who will have to bear its cost. Save time to also perform it on your end, and force the researcher to re-collect or confirm the data points that look questionable. If a company refuses to do so, you are getting bad customer service. Nobody likes bad customer service.
Bottom Line: Everybody talks about ‘big data’, seemingly without worrying about the quality of that data. Of course, ‘big data’ typically refers to data sets much larger than what we discussed here, but we all know that without top-notch input, every analysis is meaningless. Even worse, it could lead you to take actions based on flawed or biased evidence and cost you a lot of money in the long run, which is the opposite of what high-quality research should allow you to do: make sound, logical, data-driven decisions that will increase your efficiency and strengthen your market position. The growth of some markets in foreign countries means that we have to go out of our comfort zone and create business relationships with local partners. While the learning curve to gain experience is steep, you can avoid the basic mistakes we just described. It also means that it is worth cultivating relationships with vendors that you know are reliable, timely, cost-effective, and with a proven track record within your organization. Then you can focus on what really brings value to your company.

No comments: