Fact-checking Consumer Reports and its 2017 VIZIO M-Series Review
January 5th, 2018Despite its pretensions of factual and technical accuracy, I’ve long had issues with the way Consumer Reports magazine manages its reviews. A notable example is the curious way in which notebook computer battery life is calculated. When CR first reviewed Apple’s 2016 MacBook Pro with Touch Bar, it wasn’t recommended due to reported battery life inconsistencies.
Apple quickly responded and, working with the magazine, found that the CR’s test methodology, requires turning off browser caching, thus using Safari’s Develop menu in macOS Sierra. It helped trigger a rare bug that virtually nobody would see under normal use. But Apple fixed the bug anyway, and the magazine reversed its non-recommendation. Still, nobody expects the MacBook Pro to deliver over 13 hours of battery life over regular use.
Without going into detail, when I’ve checked the reviews of cars I’ve owned and/or driven I’ve run across occasional stark differences in the way features were described. The thin side bolsters of the leatherette seats of recent VW Jettas, for example, are not thin but thick and really holds you in place around a sharp curve. Or perhaps there was a manufacturing change of some sort after CR did its review.
So we come to a section containing TV reviews, and I decided to compare CR’s rating to what I achieved in evaluating a 55-inch 2017 VIZIO M-Series. Whereas the set has received really high scores from CNET and other sources due to its combination of high performance and a low price, CR’s rating was a decidedly mediocre 62 compared to similarly-sized models that earned scores of up to 88. To be fair, the highest rating includes some decidedly pricier sets including OLED models from LG.
Now in comparison with other reviewers, CR buys all the sets it covers anonymously from regular retail establishments. Supposedly it makes the publication incorruptible because the manufacturer can’t send a ringer, a specially adjusted sample that would deliver better performance than the shipping product. Not that I ever encountered anything of the sort in the 25 years I’ve been reviewing tech gear, but I’ll grant is may be possible.
Regardless, CR is entitled to its priorities, but not its facts.
Just where did the VIZIO fall down? Well, it’s downgraded in several areas, such as HDR “effectiveness,” and the supposedly “limited viewing angle” that actually scores as “good.” Go figure!
In addition, the lack of a TV tuner is criticized, but there is also a peculiar conclusion, that the “required tablet-control device [is] not included.”
I understand where the lack of a tuner might be important for some people. This design decision is clearly intended to cut costs and not have people pay for a feature they aren’t going to use. Tuners are available from Amazon for around $30 or so, and thus it shouldn’t be an issue. In the future, there will be an updated broadcast standard, ATSC 3.0, supporting 4K and other features. So when they arrive, you’ll be able to buy one without being saddled with a TV that has the older hardware. I suppose that’s a way to future proof.
The claim that a tablet is required is simply not true, although a mobile device will help expand the built-in Google Chromecast feature. As it stands, the set ships with a small number of preloaded apps that include Amazon Prime Video, Hulu and Netflix. This selection probably accommodates most users. For those who want YouTube and thousands of other services, you can pair the VIZIO with an iOS or Android device with the company’s SmartCast app.
No tablet required. On the other hand, since the CR review is based on an older firmware version on its test VIZIO, perhaps these apps were added in a subsequent update.
CR also claims there is no Internet capability, but since the set offers both an Ethernet port and Wi-Fi connectivity, and uses the Internet to receive streams for its embedded apps, the statement is just not true. Thus the review is a little jumbled. Maybe CR is reviewing so many sets, it just can’t get all its facts in order.
In other review categories, Ultra HD (4K) performance is rated as “very good” largely because of less-than-stellar upconversion from HD-to-UHD. Since most of what you’ll be watching on such a set is HD, this process is of critical importance. For me, cable reception is clearly better than on my previous TV, a 2012 VIZIO E-Series, but edges, particularly lettering, are sometimes jaggy if you look real close. Otherwise it’s not going to be much of an issue.
That said, I was particularly interested in the “Optimized Picture Settings” that were obtained in CR’s test laboratory. In brief, they were actually quite close to the Calibrated Dark settings achieved by CNET. Compared to the Calibrated setting, the backlight is turned way down. Consistent with other reviews, sharpness is set at zero. It’s generally felt that set manufacturers make the edges just too sharp, perhaps to make a better impression when customers do comparisons.
So I switched to the CR settings, which took maybe a minute or two, and turned off Auto Brightness, which I had been experimenting with. So far there is a definite if slight improvement in color rendition, particularly flesh tones. That’s probably the result of a somewhat higher color setting, plus changing the set’s gamma from the default 2.2 to 2.4.
Overall, the CR review seems fairly consistent with some of the results I achieved, but I’m concerned at the contradictions and clear errors in some ratings categories. This demonstrates a lack of attention to detail, or perhaps the editors are so reliant on boilerplate templates for reviews that they were a little careless in the final editing process.
Regardless, as my review progresses, I continue to enjoy the rare selection of true 4K content that reveals the sets superior picture in all its glory, and even the standard HD fare from the cable company looks a whole lot better.
Print This Post
