As most of you know, I’m no fan of Consumer Reports or the way it tests tech gear and other products. Sure, the magazine presents itself as superior to other publications because tested products are purchased at retail, usually anonymously, rather than provided free by manufacturers. But that doesn’t mean the test methodology or the results shouldn’t be questioned.

After giving the iPhone X a pretty favorable preliminary review, Consumer Reports found reasons to downgrade the product in its final report. Indeed, one of those factors appears to reverse a conclusion from the original test, but it’s fair to say that CR worked really hard to find faults in the iPhone X, just as they appear to do with other products under test.

In brief, the Samsung Galaxy S8 remains number one at a score of 81, largely because of its superior rating for battery life. When it comes to durability, however, it doesn’t fare so well. CR rates it as fair, largely because of a broken display after 50 drops in a device they refer to as a “tumbler,” which, of course, has nothing to do with the name given to the Batmobile in the super hero film, “Batman Begins.”

So CR recommends that, “a protective case is a must have.”

To be fair, dropping a smartphone two-and-a-half feet 50 to 100 times onto a hard surface is a pretty extreme maneuver, and a tech gadget has to be pretty tough to survive such abuse. So it’s not surprising to see the Samsung end up unwell as a result. It’s a factor, however, that should result in the product being downgraded.

While CR has other priorities, I would think that the Galaxy S8 should have lost at least a point or two as a result, even if the battery life is superior to other smartphones.

On the other hand, the iPhone 8’s = fared a lot better in durability testing, even though it was downgraded for shorter batter life. Says CR: “After 50 drops in the tumbler, our experts rated it as excellent.” It survived another 50 tumbles, and ended up in “good working condition.”

The iPhone X evidently had shorter battery life than its cheaper cousins. Two of the three units CR bought for testing suffered screen damage after 50 drops. Another suffered serious damage to the rear glass after 100 tumbles. So CR also recommends that you buy a case for this model, but it is not at all clear if the units still worked even after suffering all that damage.

Now in its preliminary test of the iPhone X, it did survive several drop tests without suffering serious damage. It took 50-100 drops to break it.

To me, it appears that the Samsung didn’t do any better in the drop tests, but the iPhone X was downgraded more, I suppose, due to shorter battery life. It ended up in ninth place.

Now under normal use and service, I doubt that the iPhones or the Samsung smartphones would be especially vulnerable to serious damage. Well maybe if you use them for frisbees. But it does seem that CR is more focused on battery life than most other factor in its reviews. This may represent the priorities of some people, but the iPhone X also had the best camera, which ought to count for something.

CR is certainly entitled to its test priorities, although it would be nice if they actually provided more information on how test results are weighted towards a final rating. You can only get that my inference.

That said, I am not going to doubt that the iPhone X doesn’t fare as well during extreme durability testing as the other iPhones. It’s the first version of a brand new design, and perhaps Apple will take this into account in developing new versions.

Consider the past.

When the iPhone 4 was dinged by Consumer Reports and a number of customers for poor reception if you held it the “wrong way,” even though other smartphones suffered from similar problems, Apple took notice. Well, after Steve Jobs said to hold it differently in an offhand comment. But he also called a press briefing where he talked about the laws of physics and allowed some tech reporters to tour Apple’s antenna design facility. For a few months, they even gave out free bumper cases. Antennagate was soon forgotten.

With the iPhone 4s, Apple changed the antenna design to something roughly similar in concept to the diversity system used in motor vehicles, resulting in improved reception under a larger variety of conditions.

The iPhone 6 Plus was criticized for being a little too easy to bend if left in your back pocket. CR went to work to see if the product was defective, but it survived their test process. Either way, the results proved to be a viable attempt at hit bait. Since it took extreme bending to damage the unit severely, bendgate was soon forgotten.

But with the iPhone 6s Plus, Apple added bracing and used a stronger aluminum, thus resulting in greater resistance to bending. Even though the original design was judged to be durable, Apple clearly wanted something better, something stronger.

So don’t be surprised if there’s a stronger iPhone X come 2018.

This article was posted on Wednesday, December 6th, 2017 at 12:00 AM and is filed under News and tagged with: Apple, Batman Begins, Batmobile, bumper case, Consumer Reports, CR, Iphone, iPhone 4, iPhone 4S, iPhone 6, iPhone 8, iPhone X, Samsung, Samsung Galaxy S8, Steve Jobs.