Friday, January 6, 2012

Why the J.D. Power IQS Has a Low IQ

by David P. Carlisle


First off, let me state for the record that the Power IQS gives me a great deal of comfort. I grew up on a farm where my family was large and money was very tight. So, it was quite natural for me to walk away from that with a significant dose of financial insecurity and concern. Fear of failure. The mere fact that the Power IQS is so successful and the OEMs pay a fortune for it shows that no matter how bad it gets, America is indeed the land of milk and honey. There is a vast market here for incompetence. What me worry? … Don’t you feel better already?

At the end of 2011 I bought a Chevy Volt – hey, I’m from Massachusetts. (By the way, the car is breathtaking – my favorite ride out of our stable of too many cars.) Right before Christmas I received the J.D. Power New Vehicle Quality Survey in the mail. Normally, I trash their surveys because I simply do not have the time to waste filling them out – and I generally love taking surveys. This time I looked at it carefully before sending it to our blue recycling tubs.

It was four pages long and asked me to make 281 (give or take) decisions on my new Volt. I didn’t even get the dollar bill they usually stuff in the envelope. I thought about this and had a bunch of questions.

Can “for profit” J.D. Power claim cult status, like non-profit Consumer Reports, so that they have “invested” survey respondents? No. Consumer Reports is a non-profit consumer “cult” group that is deeply invested in the information and the process. One million of them take the survey each year:
“Every year, the Consumer Reports National Research Center sends the Annual Questionnaire to all qualified, paid Consumer Reports and ConsumerReports.org subscribers. It is one of the largest and most complex consumer surveys in the world. In fact, other then the U.S. Census Bureau American Community Survey, the number of AQ respondents may be larger then the sample of any other survey conducted in the U.S. We received responses from more than one million subscribers, who provided information about many products and services, including about 1.3 million vehicles.”
(http://www.consumerreports.org/cro/consumer-reports-national-research-center/annual-questionnaire/index.htm)
The Power IQS “cult” is more about the OEMs who buy the surveys, and their purchased rights to proclaim victory, than it is about who actually takes the surveys. Car and Driver, a magazine that resonates with automotive “cultists”, seems to be falling out of love with IQS. Maybe a few of their staffers tried to complete the survey and figured out that it was pretty awful.
“Car and Driver paints the complaints that have dogged Ford among the latter, suggesting that these are less valid quality metrics than something like a poorly assembled door panel. The venerable buff book doesn't stop at defending Ford, however, cautioning against what it sees as a trend "toward the lowest-common-denominator solutions to reduce design 'problems.'" Citing examples like BMW moving its cruise control from a separate stalk to buttons on the steering wheel to appease whining customers and Porsche getting dinged on IQS for using brake pads that generate too much dust, C/D doesn't say it outright, but certainly implies that J.D. Power is no longer pushing automakers in the right direction.” “(http://www.autoblog.com/2011/12/05/car-and-driver-calls-out-j-d-power-iqs/)
I understand Consumer Reports’ subscribers filling out their surveys as if it were a civic duty, but why would these folks invest another 23.4 minutes into this IQS survey? I don’t think so. My guess is that these two groups are mutually exclusive. And, I think I know what kind of person takes the time to fill out a CR survey. IQS? I’m stumped.

Golly, well, then, just who fills out these surveys? OK, I’ve got to make 281 decisions. Decisions like figuring out if there was an engine idle problem at the first start of the day? What about at other times? (two decisions to make here). Or, did I get a security/alarm system? Was it factory or dealer installed? Was it aftermarket installed? Hmm. Got me. Is OnStar both a Telematics system and security/alarm system? I honestly don’t know. Did I get a “Homelink” system? What’s Homelink? At five seconds of thought for each “decision”, it would take more than 23 minutes to complete the survey. Got me thinking - what is the profile of someone who invests 23 minutes on a paper survey, stuffs the envelope, and takes the time to mail it? They do have an “about you” section on the IQS survey. For example, you could check the boxes for being a 45 year old five-and-a half-foot female weighing 135 pounds with two less-than-20 year old kids at home, making $150,000 a year. My big question is, does this someone really represent her economic/demographic group? Someone who would take 23.4 minutes (excluding the time to mail the damn thing) out of their busy day to fill out the IQS survey. Just think about it. It is ridiculous.

Do the respondents really understand what they are being asked? This is a relevant question that the OEMs should be asking. OK, so I just bought a new Volt and everybody in our family loves it – husband, wife, and two daughters. Four people in the family drive it, and we got the survey before the first 1,000 miles were put on it. The letter from Finbarr J. O’Neil says that the survey should be filled out by the person who drives the vehicle most often. That could be any of us. Lets go to page one:
  • What transmission type is your new vehicle? I think it is a CVT; the others in the family are clueless.
  • Do we have auto temperature control? I think so; the others in the family are clueless.
  • Does it have side airbags? Well, I certainly think it should have; the others in the family are clueless.
  • Does it have a Homelink system? We are all clueless.
  • Does it have tires and rims installed at the dealer? Or, the aftermarket? I know that the car came with these; the others in the family are clueless about what the term of art “aftermarket” really means. (Most dealers source accessories from the aftermarket … if I were to know this, how should I answer the question?)
At the end of page one I’m getting tired and confused. Of the 281 decisions I’m asked to make, I’ve just handled 29. I’m about 1/10th of the way through the survey. Page two requires me to make 161 decisions on things like the airbag/SRS light being broken, or in a poor location. Like getting wrong directions from my navigation system. Like my hands-free communication not recognizing a command. Or, the tire pressure monitoring system being broken? Three fourths of our family doesn’t know what an SRS light or tire pressure monitoring system is, and have never had any luck with navigation systems, so we just don’t use them. Inside of five minutes my younger daughter could figure out how to use a Tata navigation system programmed in Hindi. Finally, none of us talk to the Volt, though it is quite articulate. It is not that these questions are bad. It’s that they are largely irrelevant.

Ask yourself just three simple questions. Lets say you’ve put 700 miles on your new car with an automatic transmission … (1) Have you even used the parking brake once? (2) Do you know if won’t hold the vehicle? (3) Have you noticed excessive or uneven tire wear in those 700 miles? Or, do you just think that most of these questions are irrelevant? How carefully will you make the 281 decisions that the IQS survey asks?

Bottom Line: Years ago J.D. Power was useful in merchandising quality differences that existed between Japanese and Domestic cars. This “merchandising” was very effective in the media to establish consumer awareness of the Power report card. Toyota, Honda, Nissan got “A’s” and the Domestics flunked. That sure was easy to understand. It was a battle between cars that worked and cars that didn’t work. The Domestics lost gigantic market share … but, after a decade or so, they finally overcame the quality shortfall. Now J.D. Power is pushing a different sort of battleground merchandising. Now the war is between cars that have flawless infotainment systems and those that ain’t got them. This is supremely irrelevant in today’s market with today’s customers.

So, what's the real bottom line? Consumer Reports should carefully license use of its automotive survey results to the OEMs. CR's survey results are an inarguably valid representation of its more than one million-strong subscribers who take product quality very seriously. This should put the J.D. Power IQS out of our misery.

No comments: