Consumer Reports Seeks Versatility
January 7th, 2015Although Consumer Reports magazine is supposed to be incorruptible — mostly because the publication doesn’t accept ads and tested products are purchased rather than manufacturer review samples — there are clear problems with the way they are evaluated. As I’ve stated though the years, Apple gear isn’t always treated fairly, largely because of curious test standards that don’t always accurately reflect what the customer should expect.
That carries through to other product categories. So the magazine’s highly valued surveys of motor vehicle reliability are not taken from dealer repair shops, independent or otherwise, but from consumers checking a few boxes on annual questionnaires. I’ve received them myself, and the information requested is extremely superficial. It’s being asked of regular people, not automotive experts, but it’s used as the basis to assert whether your car is reliable or not. Sure, a car prone to have engine, transmission, suspension and audio system repairs deserves to be rated unreliable. It doesn’t require an auto expert to know whether you had to bring your car into the shop for a serious problem. But many of the finer points are not being accurately considered; again, because the survey is unscientific and isn’t specific enough to provide a comprehensive picture of how well your car will fare under normal use and service.
CR is particularly unable to deal with tech gear. When it comes to smartphones and tablets, for example, products with more features are apt to get higher ratings. It doesn’t matter if some of those features are barely functional. Samsung is one of the worst offenders, and I’ve already covered the Galaxy S4, the king of junkware-ridden handsets, and the Galaxy S5, which sported a barely functional fingerprint sensor.
When it comes to personal computers, CR has devised a category known as “Versatility” with which to evaluate some of the features offered in a PC. According the magazine’s annual buyer’s guide for 2015, “Versatility includes hardware such as memory-card slots and A/V connections, software such as security programs and productivity applications, and tech support and warranty provisions.”
On the basis of this curious definition, it really doesn’t make a whole lot of sense. A product’s “Versatility” shouldn’t be judged by the length and breadth of the warranty or the quality of tech support. These two should be moved to the existing support or reliability categories.
Now it’s interesting that many of the models from Apple that CR tests are actually priced in the same range as top-rated products from other companies. But it’s also true that Apple tends to be frugal about adding lots of ports, and that’s one reason to perhaps downgrade Macs. But “security programs” is largely a non-issue on the Mac platform, so it shouldn’t be considered even though there are loads of such programs for Windows users.
Unfortunately, CR’s evaluations are so general, lacking key specifics, that the reader doesn’t know whether the items under “Versatility” are weighted based on importance or relevance, or simply checked off and given equal consideration. This lapse of attention to detail represents the larger issue with CR’s PC ratings, which is that most of the differences between operating systems are rarely explained. To the reader,the OS X and Windows are essentially identical platforms that just happen to be built by different companies.
I realize that CR is catering largely to a non-technical audience, people who just want to know how what to buy and get on with their lives. To them, the fine details of automative design and the technical niceties that separate a Mac from a PC may not be so important. You might as well be talking in a foreign language.
But this doesn’t mean such people are necessarily ignorant, or plain stupid. Some may be positively brilliant in the fields in which they operate, but they have other priorities. That’s a perfectly reasonable approach. Unfortunately, CR doesn’t serve such people terribly well, because they aren’t really being given the information they need to know which products and services might suit them best.
Worse, the magazine’s heavy dependence on unscientific reader surveys means that the reader isn’t always getting accurate information about which products are the most reliable. People with problems are generally more likely to respond to such surveys, while those who have no problems are apt to set the surveys aside and worry about more important matters.
This, too, is the reason that online reports about new products always seems heavily weighted towards the defects. People with gear that just works seldom bother. So I have a car that has never needed more than scheduled maintenance, a wheel alignment and a single battery replacement. I’m hardly inclined to concern myself with filling out surveys about my experiences with the vehicle, but if it gets a lot less reliable as the mileage climbs, you can bet I’ll have something to say.
For now, I still have an old J.D. Power survey invitation on hand, and I haven’t bothered to fill it out. Maybe I am cheating that company of the benefits of my experience, but I just haven’t had the time.
In any case, my arguments about CR are largely unchanged. The writing style is less dry than it used to be, the basic editorial quality is superior, but the reviews, despite a somewhat hipper approach, still suffer from the same shortcomings.
Print This Post
