Apple tends to release little more than bare-bones specs for most of their mobile products. Beyond basic dimensions, battery life, and supported multimedia formats, there’s not a lot to find. That requires some heavy lifting by others to determine, and sometimes you wonder if they’re looking at the wrong things.

First there’s the inevitable teardown. You get a “bill of materials” (or BOM), a summary of the parts used and the estimated prices. Now knowing that Apple buys this part from Samsung, that part from Qualcomm, might appeal to you in an abstract sense, if you care about such things. That roughly half the parts in the iPad 3, in terms of estimated price, come from Samsung, is especially curious. After all, Apple has been suing Samsung for alleged patent violations to the ends of the planet.

However, Samsung is a multilayered company, with many divisions that act almost like separate companies. So they might be making billions as a parts supplier to Apple, while the mobile handset division is fighting tooth and nail to defend their right to build more iPad or iPhone knockoffs. But Apple will buy parts where they can get the best deals, with consistent quality and prompt and predictable delivery times.

Unfortunately, you have to wonder about the estimates for the prices for parts. I’ve seen reports, for example, that Apple is paying either $70 or $87 for the Retina Display. The entire bill for these parts indicates Apple is evidently taking a decent hit to profit margins on the third generation iPad, no doubt expecting that those prices will go down over time as manufacturing techniques become more efficient.

Regardless, how can anyone outside of Apple and their suppliers know the exact prices? Apple is legendary for negotiating billion dollar deals with suppliers to get large quantities of the parts they need. The prices you read are estimates, and maybe other companies may pay those prices to get the same components, but wouldn’t Apple do better if they’re buying ten or twenty million at a clip? Offering hefty upfront payments can certainly influence those prices. The people who know the truth aren’t going to reveal any of that information except, perhaps, years later after the raw numbers no longer make a difference.

Beyond the BOM, there are benchmarks, just to see how well, say, an iPhone or an iPad fares against the competition, particularly products running the Android OS. When you read those benchmarks, you’ll find that there often isn’t a lot of difference. Maybe a few fractions of a second faster or slower in browsing speeds, or minor variations in game frame rates. If you believe only in numbers, the fact that some Android tablets have eight megapixel camera sensors, and Apple only installed five megapixels in the iPad 3, ought to mean the former delivers superior pictures.

But it’s not just the raw numbers, but in how well the camera software can turn the data into real pictures that counts. So far it seems that the latest and greatest iPad is making more of its pixels than the competition. Step up to the iPhone 4s, with eight megapixels, and you find good reason for many people to leave their point and short digital cameras in the closet. Sure, those cameras offer extra customization options, including optical zooming, which will deliver superior snapshots, at least in theory. In practice, most people won’t care, or notice a significant difference. Apple demonstrates the real advantages of their mobile computers, specifically the iOS and the vast app catalog.

However, testing the advantages of one operating system over another is a far more granular process, involving the weighing of numerous pros and cons that can be time-consuming to evaluate. It’s a lot easier to say that Product A is better than Product B because the specs are better. That’s why Consumer Reports wants you to believe that smartphones with larger screens must be superior to the iPhone, even though there are well-known usability concerns. But determining usability is not something you can do with a pocket calculator or with bullet points that emphasize product measurements.

Of course, the subjective factor isn’t restricted to computers, operating systems, and apps. Consider a speaker system. You can find lots of gear with roughly similar specs, but actually measuring the performance of loudspeakers in your listening area is a highly complex affair. Room reflections, not to mention the source material (rock, classical, folk, country, etc.) can highly influence your reaction. I recall doing both measurements and listening tests with computer speaker systems years ago for Macworld. It took weeks, and ended up being a far more complicated evaluation than the magazine would have undertaken had I not insisted on following the testing model of some of the audio magazines of the period. But most people just listen to decide which products they prefer.

Obviously customers rate the iPhone and iPad superior to the competition. User experience may be quantified on a spreadsheet, dividing one’s reaction into different categories, such as interface response, ease of use, and so on and so forth. But often it’s a magical combination of all these factors that comes together and makes one product feel superior to another. The snappiness of the touch interface on an iPhone and an Android smartphone are clearly different, though less so than it used to be. But can you quantity those differences with raw numbers, or one’s subjective impression?

In the end, specs and benchmarks are just easier to do, but they will often give you only a vague idea of how a product performs in the real world. Doing it the right way, however, isn’t easy. Agreeing on the proper method is difficult too, which is why so many product reviewers choose to take the easy way out.

This article was posted on Tuesday, March 20th, 2012 at 12:00 AM and is filed under News and tagged with: Android, Android OS, Android tablets, Apple, bill of materials, Consumer Reports, Digital Cameras, iOS, iPad, Iphone, Macworld, Operating System, Qualcomm, retina display, Samsung, smartphone, speaker.