Consumer Reports Does it to Apple Again
October 18th, 2017You can bet that, when reviewing smartphones, Consumer Reports magazine appears to have a blind spot towards Samsung; maybe a few blind spots. How so? Well, I’ll get to that shortly.
Now on the surface, CR ought to be the perfect review source. Unlike most other publications, online or print, it actually buys tested products from retail stores. That includes luxury cars costing over $100,000 if need be. So, in that area at least, it should be incorruptible. Compare that to regular publications that contain reviews, most of which receive free samples from the manufacturers.
Indeed, when I announced recently that VIZIO sent me a 4K TV for review — with no preconditions as to how I rate the product — I got a comment from a reader suggesting that my article would somehow be tainted. But I’ve been reviewing tech gear received on that basis for over two decades, and it’s definitely not a factor. Never has been.
But even if there’s a tiny bit of suspicion on the part of some people that product reviews might be slanted if those products are sent free of charge, I am not surprised that CR gets high credibility. So there’s a story from Seoul, South Korea touting the fact that, “Samsung’s Galaxy S8 tops U.S. consumer review.”
South Korea? But isn’t CR an American magazine? Yes, so this story no doubt originated from Samsung, even though a manufacturer is theoretically prohibited from quoting a CR review. So the article mentions the conclusion, not the contents, so even if it was originated from Samsung, the company is off the hook.
According to the latest CR report about smartphones, the Samsung Galaxy S8 and the Galaxy S8 Plus gained top ratings by CR. Number three, peculiarly, was last year’s Galaxy S7. Really. So where did the iPhone 8 end up? According to CR, fourth and fifth. Number six was the Galaxy Note 8.
I decided to take a look at the factors that put the iPhones below three Samsungs, including one of last year’s models. Let’s just say it didn’t make a whole lot of sense in the scheme of things, but I’ve had these issues before with CR.
Take, for example, the Galaxy S8 versus the iPhone 8. The former is rated 81, the latter is rated 80. So despite the implications of the article from that South Korean publication, the scores are extraordinary close. A minor issue here, another minor issue there, and the results might have been reversed.
But was it it that makes the Samsung ever-so-slightly superior to the iPhone? Unfortunately, the two reviews aren’t altogether clear on that score. So on the basis of 11 performance categories in which the two phones are rated, the iPhone 8 has six excellents, four very goods, and one good. So in theory the Samsung should have scored better in these categories. However, it has four excellenst and seven very goods.
From my point of view, the Apple ought to rate better. More excellents, right? But there is a Good rating for battery life, whereas the Samsung rates as Excellent. Evidently that factor must supplant all other considerations and award the Samsung with a higher total. Curiously, the longer battery life of the iPhone 8 Plus evidently didn’t merit a rating higher than Good either. Just saying.
But there’s more. It turns out that the iPhone is far more resilient to damage than the Galaxy S8. According to CR, the iPhone “survived the water dunk test and our tough 100 drops in the tumbler with just some minor scratches.”
Evidently, being a rugged mobile handset doesn’t count for very much, because the qualitative ratings don’t include that factor. So the Galaxy S8, according to CR, doesn’t fare nearly as well. The report states, “The screen is rather fragile. After 50 rotations in the tumbler, our experts rated it only fair. The display was badly broken and not working. For this phone, a protective case is a must have.”
What does that say to you? It says to me that the Galaxy S8 should have been seriously downgraded because it’s very fragile; users are forced to buy extra protection for normal use and service. Smartphones are routinely dropped or knocked against things.
To me, it’s barely acceptable. To CR, ruggedness doesn’t matter.
Nor does the reliability of a smartphone’s biometrics count evidently. As most of you know, the Galaxy S8 and its big brother, the S8 Plus, have three biometric systems. The fingerprint sensor, located at the rear, is an awkward reach. You are at risk of smudging the camera lenses instead. Both the facial recognition and iris sensors aren’t terribly secure. Both can be defeated by digital photographs.
In short, you have a breakable smartphone with two biometric features of questionable quality being judged superior to another smartphone that’s rugged and has a reliable fingerprint sensor. But maybe it has somewhat shorter battery life than the competition. In other words, CR seems to regard battery life above other important factors, but how ratings are weighted, and why potential breakability is not considered, is just not mentioned.
But since CR buys the products it reviews, the serious flaws in its review methods aren’t important. The media that continues to quote the magazine’s ratings without critical comment aren’t helping to encourage CR to change its ways.
And please don’t get me started about the curious way in which it rates the battery life of notebook computers.
Print This Post
I gave up subscribing to CR many, many years ago. In my opinion, they have their minds made up before they do any testing. Apple is only one of these.