Can One Magazine Review Vitamins and Personal Computers?
August 10th, 2010It seems scarcely an issue of Consumer Reports appears that doesn’t cover personal computers and other electronics gear these days. In the wake of their strangely-concocted criticism of the iPhone 4’s antenna design, CR has gotten loads of favorable public attention — often not deserved.
It’s unfortunate that virtually nobody in the mainstream media ever seems to take CR to task for its serious shortcomings. Yes, the magazine doesn’t seem to be influenced by manufacturers, because it is published by a non-profit corporation that doesn’t take advertising. What’s more, the tested products are all bought at retail at regular dealers. That includes autos, which means that CR must run up bills in the millions of dollars to acquire all these products.
At the same time, being perceived as incorruptible doesn’t mean CR can’t be wrong. As I’ve said before, being a jack of all trades and a master of none has its shortcomings. CR wants to reach a general interest audience that is perceived as not technically savvy. So they strive to oversimplify technical details in such a way that the interests of would-be buyers isn’t being served.
In the September 2010 issue, food supplements and vitamins share top billing, followed by computers, printers, smartphones and even 3D TVs. You also learn “5 Ways to Slash Your Cell Phone Bill.”
Is there anything they don’t know?
Well, yes, because they seem totally ignorant of the fact that the most popular personal computers are available with either of two different incompatible operating systems. The latest review does mention that there is something called Windows, but the existence of a Unix-based alternative, Mac OS X, isn’t part of the picture.
While Apple continues to rate high among the product ratings, usually at or near the top, the subjective category of “Ergonomics” restricts its focus to such hardware considerations as “the quality of the keyboard and mouse or touchpad and accessibility of features.” The usability, predictability, reliability and performance of the operating system isn’t part of the picture.
A magazine that prides itself on trying to protect the consumer ought to also add a category covering a computer’s security. After all, billions of dollars are lost almost every year as the result of Windows-borne malware. But CR doesn’t appear to want you to know that it’s essentially a Windows problem.
Indeed, the average reader who isn’t well versed on such not-so-fine distinctions would come away with the impression that Macs may be highly-rated, but they are also expensive, so why not just select something cheaper since it ought to be near as good?
In addition to eliminating one of the key considerations in the choice of a personal computer, CR fares hardly better with a brief survey of tablet computers. Yes the iPad seems to get the highest marks, but its gaming performance is, peculiarly, rated as simply average. An unnamed $700 laptop gets a “better than average” gaming score, which is preposterous. Note-books in that price category generally have integrated graphics or the cheapest possible discrete graphics chips. Many popular 3D games are simply too powerful to function on such hardware at high screen resolutions with all or most options selected. Or maybe CR relies strictly on Pacman.
In contrast, every single game on an iPad is not just playable, but performs exceedingly well according to every test report I’ve read.
Now I suppose some might suggest that CR has to be correct because their gaming test methodology is more scientific. But we don’t know what that test methodology is, since it’s not mentioned in the magazine. More to the point, I challenge CR’s editors to produce a $700 portable computer that can play all the popular PC games at a decent frame rate, if they run at all. In contrast, I also challenge CR to tell us which iPad games fail to deliver good performance, assuming that there’s no program bug that might create a playability problem.
On the positive side of the ledger, CR does seem to make a passing effort to define the various smartphone operating systems, although the claim that the Android and iOS are very closely matched is not borne out by most reviews of these products. Or maybe CR’s standards, whatever they might be, are far too lax to accommodate anything but a drastic difference.
However, the iPhone 4 does seem to fare well, simply because that absurd laboratory test of its antenna system isn’t mentioned. Perhaps the magazine’s deadline passed, or CR is starting to realize that junk science doesn’t trump Apple’s multimillion dollar antenna test facility. Or, for that matter, those widely-publicized videos that demonstrate that rival smartphones are also prone to signal degradation when held the “wrong” way.
At least I now know which multivitamin to purchase, and which food supplements to avoid. I feel confident that I’ll live a longer and healthier life as a result, although I’ll continue to remain in total ignorance about what electronics gear to buy, at least if I depend on CR for that information. And I don’t.
Print This Post
Not so sure I would rely on CR (or ‘Which?’ as in the UK) for an judgment calls at all.
My reasoning is that in my own few fields of expertise, I am able to see how mistaken and misinformed they can be.
Why would that lead me to believe they are not as bad in areas which I don’t know about?
[…] | Print This Article […]
Some of their information is quite valuable, however. Their reliability ratings for cars are a good place to start shopping. Of course they take no consideration of style, features, fun-to-drive or performance, but knowing which models are more likely to be in the shop is good information.
I would imagine the same information would be quite valuable for other large purchases like flat screen TVs or refrigerators, and CR is one of the places I would go for that information.
As a car enthusiast, I wouldn’t trust CR’s opinion of which car was better, but I would trust them (and others like JD Power) to anticipate reliability, which is one of many factors I consider.
That relates to computers because like cars, there are just too many variables for a magazine like CR to address. Even eliminating the OS issue, what about video cards? CR’s simplistic look at Windows machines wouldn’t go into much detail about the gaming performance of a video card, but would go into reliability ratings of Brand A vs Brand B. As a savvy shopper, I’d pick what I want first, and then go to CR to see how it fares. CR can’t pick the best product, but it can help avoid picking a lemon.
@Andrew, THe problem with their reliability ratings is that they are evidently totally based on unscientific reader samplings. When you subscribe, you get a survey in the mail. They do not pick or choose which readers get the surveys and how they conform to the demographic makeup of the U.S. population as a whole. I suppose that such a survey may target seriously defective products. Otherwise, I’d consider them with caution.
Peace,
Gene
@Gene Steinberg,
But things like “Has your refrigerator needed repairs?” aren’t too subject to which users are querried. If 20% of Brand A had problems with the ice maker and only 5% of brand B had the same problem, then it stands to reason that Brand B has a more reliable ice maker.
Same with cars. If 35% of 2004 model A had electrical issues but only 10% of the 2003 model, that would clearly indicate something being up with the electrical system on the 2004 cars.
Likewise with computers. iBooks had far more problems than PowerBooks of the same age, reflected in reliability stats.
@Andrew,
Maybe. But if only 3 people reported on the 2004 model and 20 reported on the 2003 model, which percentage is more reliable? And which part of the electrical system had a problem? How many of the readers reporting even know what is electrical and what isn’t? Seems like maybe it is just common sense, but todays cars are complex beasts. It isn’t quite as cut and dried as it once might have been.
And consider this when reading reader reports. Back in the day (and I’m old enough to say “back in the day”) when I was doing a lot of work with the auto industry, Jaguar went from lousy reliability to great reliability. If one did the research, they found that reliability hadn’t really changed. But what had changed was how Jaguar handled the problems, and thus the owners’ perception of reliability.
Mac users should know this syndrome, too. If someone buys something that is supposed to be reliable and doing so meant they are smart, when something goes wrong, it isn’t as big a deal as when someone less inclined to be forgiving has something go wrong with the same thing. It is about expectations.
“I own a Honda because they are great cars.” “Hondas are great cars!” “Yeah! I love mine!” “No problems.” “Right, no problems (forgetting/ignoring the time when I had to drop $1000 to get the transmission fixed one month after the three year warranty died, or the $250 I had to spend to get the oil pan replaced, etc., because I was supposed to be smart to buy a Honda since they are great cars.)
Another way to look at it, back in the day NYC had a sudden drop in crime. NYC celebrated. What was discovered was that it wasn’t so much a drop in crime as it was a drop in reported crime. In other words, it was a matter of being mugged in NYC was just part of being a New Yorker. Problems are only real with CR when people report them. And people only report them to CR when they think they are real. Perception is reality far more in CR than people realize.
So let’s relate that back to your example. What if nothing really changed between the 2003 model and the 2004 model? The problem with the CR model is that it doesn’t really address if there was a problem, really, what the problem was (“electrical system” is quite broad these days, some issues are more severe and expensive than others), why there was a problem, or how the people affected by the problem dealt with it.
So all in all there is a lot to distrust about CR regardless of what is being examined.
Joe
I was a long-time subscriber to the CR magazine. When the World Wide Web came along, the magazine was dropped and went online to subscribe. I stopped the online subscription about five years ago. It was great for consumer information and for ratings on household items and stereo equipments. Too many times I could not find their highly-rated stereo components in my local market. I ended up making my purchases based on reviews in stereo magazines. This made me look at CR as a guide and info to make a purchase which made not be the one in the ratings.
My Mac purchase did not involve any CR review. It was because of using Windows at work that pushed me to get a Mac for home. I paid more, but never had any regrets. I am on my third Mac today and looking forward to the next one late this year or early next.
I read a CR review on computers around eight years ago. It was then the realization that CR did not know how to evaluate one properly. It left a bad taste. Over the years, I moved on and used other sources to make my purchases.
Gross problems yes. Fine distinctions no.
Peace,
Gene