Memo to Apple: How About Better Release Notes for Public Beta Testers?
July 29th, 2015An unknown number of beta testers are now eligible to receive prerelease versions of OS X El Capitan and iOS 9. So far, there have been three releases of the OS X public beta and two releases of the iOS public beta, compared to five betas of El Capitan and four betas of iOS 9 for registered developers. The process is clearly moving quickly, and it’s reported by some developers that beta five fixes a number of serious glitches with the next OS X.
But Apple doesn’t make it easy for developers or public beta testers. Members of Apple’s developer program can download a sparse set of release notes for each update, but those receiving the public beta receive little or no information about the state of the release. There is no comprehensive listing, at least outside of Apple’s developer team, indicating what was fixed and, except for a few serious issues, what still needs to be addressed.
It’s not that Apple doesn’t make it easy to report problems. There’s a Feedback Assistant app for public beta testers, and a developer’s have a place to report bugs. It’s a one-way-street for the former, generally with no response from Apple. Developers will reportedly hear from Apple when more information is required.
Developers may receive information about the things that ought to be tested in a forthcoming release, but again it’s all delivered in a very general way, without the necessary specifics to see why those things ought to be tested.
I’m of mixed minds about the value of the public beta. While reaching a wider range of testers is a good things, a lot of people don’t take it seriously enough. They will reveal problems to others, and sometimes even review the prerelease product and criticize it because of the bugs you expect in beta software. One blogger even attacked Apple for poor quality control because of the reports of problems with El Capitan. I don’t know if I should laugh or cry.
I realize Apple wants to make the beta test process as simple as possible for the public. So the warnings are brief and the non-disclosure cautions are basic, although Apple points you to a longer document. In short, if Apple has publicized a feature, it’s all right to mention it to people outside the beta program. Apple has released a decent amount of information about both El Capitan and iOS 9, so you have a fair amount of leeway. Otherwise don’t say anything about it, which means don’t write a review, and don’t post your opinions on your Twitter or Face-book accounts, as common examples.
And be sure to have a recent backup at hand in case things go wrong and you need to restore your Mac or iOS gadget.
Now it’s not that Apple is apt to clamp down on public beta testers unless they actually post the prerelease software or screenshots. Developers who break the rules, however, may find themselves tossed out of the program, and Apple would have the perfect right to take appropriate action.
Regardless, I see message boards on Mac-oriented web sites with all sorts of information about the beta process, including long lists of fixed bugs, and the bugs that were discovered or haven’t been addressed. Technically, the testers are speaking out of turn. On the other hand, maybe Apple should take advantage of all that interest, and set up a public beta discussion board for testers to exchange views. Assign a few moderators to make sure things don’t get out of hand, and maybe even respond to frequent reports of problems.
It’s not that Apple can keep any of this out of the hands of competitors. They are all free to join the developer program, which costs $99 per year, or sign up for the public beta. Nothing about the new operating systems is therefore a secret to anyone. It’s all out there, and Apple might profit from taking advantage of all that attention.
At the same time, there ought to be a central repository listing the status of known problems, so it’s easy for any tester to figure out whether they are all alone in their grief, or if Apple is aware of the bug and is working on a solution.
Unfortunately, the Yosemite beta program didn’t make me feel warm and fuzzy about the process. It should have resulted in a more stable release, but it doesn’t seem to have worked out that way. Don’t forget the Wi-Fi bugs that essentially persisted until the 10.10.4 update was released a few weeks ago. I can’t believe for a moment those problems weren’t noticed and reported by testers last year ahead of the original Yosemite Golden Master (I never had the problem). It doesn’t matter, really, why, only that Apple should have been aware of the problem way before Yosemite was finalized.
One key reason for this unfortunate lack of information is Apple’s penchant for secrecy. That makes sense when it’s about the next iPhone, the possibility of an iPad Pro, the feature set of the next Apple TV, or something wonderful in a whole new category. But when the operating systems are free, and just about anyone with an Apple ID can get the prerelease version, getting a full picture about the state of the beta product and it’s progress isn’t going to give away the goods to the competition. They already know what’s going on.
Print This Post
Thanks for this, Gene. You’re putting into words something I’ve been feeling for sometime. it’s fun (and actually not very risky) to get a sneak peek at what Apple has in the works. But beyond that I don’t have a clue what we testers are supposed to do. And on the few occasions I’ve been moved to submit any feedback I’ve never had the sense that anybody’s listening. all in all there’s something a bit strange about the whole program. Also, Apple isn’t big on market research, and I’ve never heard of them going back on some innovation because a lot of testers hated it. That’s another kind of listening they could do better.