Simon Bradshaw

sms -n 07123456789 -m "typing command lines in T9 is fun"

I think you know Mr daveon, I'm sure he will be along shortly to comment (allowing for time zone differences).

My impression is that a lot of open source people think things are easy because they are working at a level of abstraction which has been made easy by people doing the hard stuff (clearly some of the hard stuff is done by open source, but the majority of the open source blowhards in the noosphere aren't doing hard stuff - I am trying to promulgate the term 'froth jockeys' for them, on the basis that they are providing the thin layer of froth on top of the liquid (coffee / beer)).

call -n13346 -hangup

From:

pjc50.livejournal.com

Your impression is radically different from mine; e.g. I know

mjg59 who does very hard work in getting power management to work on laptops under linux. There certainly are a few Eric Raymonds around, and I'm unimpressed by the "Ruby on Rails" lot, but I'm curious as to what you think the "level of abstraction which has been made easy by people doing the hard stuff" is.

From:

It is the 'Ruby on Rails' lot I am thinking of in particular, and if you were to look at the average open source blogger most are of that ilk. The fact that you know one person doing hard stuff doesn't mean that most people aren't.
If I had to given an example I would say 'the internet' in all its messy physical glory, there is a lot more to it than passing a few TCP / UDP packets around.

From:

Most of 'the internet' has been developed by professionals, with support from their employers.

Weenieware is something that some kid scrapes onto sourceforge and ignores the support requests for, but adds K3wl Th3M3Z0rZZZ!!!!" to.

From:

One Eric Raymond is an Eric Raymond too many. For all that he talks a good game about OSS, what has he ever actually contributed?

Man's a gunw*nk c*nt. I object to the way he runs|ran the Jargon File, even.

From:

As you may know, the original Jargon file can be found (search for Original Jargon File - the main site, dourish.com, seems to be down at the moment) before it was mutated by ESR. It is, for those of us who are middle aged and bitter and twisted, amusing that in the original things refer to LISP / Lisp Machines rather than C and UNIX, particularly because in general the complaints made by the Lisp / Lisp machine people about C / UNIX are almost identical to the complaints made by C / UNIX people about VB / Windows people.

From:

I have what I think is the last GLS version of the File. Much funnier, less political axes to grind, and, when it comes down to it, considerably more erudite. Raymond's wrecked it with heavyhanded open-source advocacy.

From:

miramon.livejournal.com

Wasn't he responsible for fetchmail? At least it has his name all over it.

From:

IIRC ESR inherited an earlier POP client, hacked the "deliver to local SMTP port" onto it and tarted up the user interface.

I've still never found a major piece of systems work ESR has done that's found a wide user base. Fetchmail is something you could hack up in a weekend, not a serious piece of software engineering. He talks a good system... if you happen to think the 'bazaar' approach is anything other than contemptible ;P

From:

Well, firstly, I'd say that getting power management to work on a mobile phone is an order or two of magnitude harder than a laptop. Not only do you have the usual suspects (applications, threads, backlights, screens blah blah) but you also have a radio circuit that needs to be on all the time, and needs to be able to quickly spin up and down if it's not being used in a wide variety of situations. It needs to regularly check the radio landscape, work out when to do handovers, check signal strengths and so forth, not to mention, you need to have a lot of logic in their to stop, for example, the radio stack doing massive high power frequency band sweeps when you go out of coverage. Tuning all of that, and getting it to work well with all the other stuff going on is a real development nightmare. Not to mention, each network, in each country is running different kit, doing different things so you need to run quite expensive test teams around the world tweaking things.

Other painful levels of abstraction... handling hardware acceleration for core codecs, for example, correctly handling the AMR11 (I think) codec for voice on a GSM phone. This will normally be a silicon codec on the radio ASIC, so you need to handle data pass through to the codec and handle how that interacts with the rest of the device, if you want to then do something like voice recording, other video with sound (H.264) you need to work out sensible ways of using the right acceleration at the right time.

Graphics... interfacing graphics to the screen drivers is a nightmare. Don't know why, but it seems to take thousands of man hours of hard graft to get working right and debug. If you then add another graphics abstraction layer on top of that it can be equally painful as you try and test and check to make sure things render correctly

Video/Audio multimedia... on dual core phones, you'll (as I've mentioned) have the audio codec running in hardware on the radio processor and video running in software or hardware on the application processor. Synchronisation of audio with video is a nightmare in those architectures. This is comon to a lot of the Marvel and TI family of processors for phones. Of course, a lot of this goes away if you stick a PC processor in and give it a Gig of memory, but that's not the direction most phone developers (with one exception) want to go in.

There's lots and lots more of those, but I'm not an engineer, I just have to tell people how much it's going to cost them for us to fix the problems when they encounter them.

From:

pjc50.livejournal.com

If you don't want one, don't get one? It's not like there's a shortage of phone designs.

Coincidentally I just read: http://www.f-secure.com/weblog/archives/00001322.html about hacking the iPhone. Basically all phones of a certain level of sophistication are general-purpose computers on which any software can be run, it's just that the openmoko phone is designed to be particularly convenient.

The network itself is fairly well protected as on nearly all phones (including the openmoko phone and the iphone) the GSM logic is on a seperate processor which communicates with the UI one over a serial link. This makes it a lot easier to get regulatory approval for the design, because it guarantees reliability of the GSM level.

Phone software reliability by big names is hardly guaranteed: My T610 crashed a few times, necessitating removing the battery and rebooting it.

People want all sorts of different things from phones. An open source phone is for people with particularly individual wants. Myself, I'd settle for a phone with a fast user interface that has no visible delay between pressing a button and moving around the menus.

From:

The real problem with the speed of phones is they really are running low end processors with really low amounts of memory, so they tend towards slow. The iPhone has the gosh wow factor on the UI for that reason - it's running a beast of an application processor with all the memory it could ever need.

The problem of abstracting the radio is you end up with poor to middling radio performance. I've read and been told by insiders at AT&T that the iPhone is ok, the voice quality is average to poor and it's ok(ish) with dropped calls, missed calls etc... Of course, it is terribly shiny so that's ok.

The other thing is it isn't price sensitive unlike every other phone in the world. So having 2 processors etc... doesn't matter. Most of the phone companies are trying to move away from that even on smart "open" platforms, and the next generation of silicon from Qualcomm, EMP etc... is based around either dual core/single chip, or single core combined radio and application processors.

As you say the GSM network is reasonably well protected, but even on devices with separate integrated 3rd party radio modules, I've seen some collosal screw ups, so it's not a guarentee, likewise, there are even worse things you can do at a application level on WCDMA with Quality of Service demands.

From:

One of my favourite wikipedia entries is that for weenieware, started by a former colleague of mine.

I want software that's written by someone whose mortgage and livelihood depends on the quality, not something tossed off by some malodorous virgin student in a baseball cap, stonewashed jeans, an allegedly humorous Perl t-shirt and a crap moustache.

High-consequence software - only trust it if it's by someone to whom it has personally high consequences ;)

From:

pjc50.livejournal.com

It would be nice if high consequences could be applied to somebody involved in this fiasco.

Generally open source works best when the user community and the developer community overlap. This is why there is lots of good open source network infrastructure software and no good open source calendaring software.

From:

covertmusic.livejournal.com

On that note, I don't get the Rails hate here. If you look at Rails and Django, the vast majority of the developers - as in, contributors to the core source-control repositories, not the many 'client developers' if if you like - are working fulltime in web-application development. Django came out of a newspapers' newsroom, actually. The two web frameworks getting all the hype *strengthen* your argument, not weaken it - the developer community is the core of the user community.

Equally true with the infrastructural stuff that came out of LJ (perlbal, mogile etc) too, come to think of it.

Sure, there are fanboys/fangirls, but doesn't everyone just channel them out? That's what Slashdot/Digg are for - fly-paper. The Django blogs anyone actually reads are http://www.b-list.org/ and http://www.simonwillison.net/ - both of which tend right towards the nitty-gritty.

And we both know doing the Web right is hard. You've read the REST book, yeah? The thing is, a lot of the complexity isn't in the code, it's in the design and information architecture. So maybe it's not "difficult" in the way hacking on the Linux kernel is, but that's just because the complexity's been moved, not abolished.

Sure, DHH, the lead developer of Rails, is spectacularly... opinionated, but that happens everywhere. In fact, the business he works for/part-owns, 37 Signals, is not backward about coming forward.

From:

I agree completely that in general the people who are working on these high profile projects usually know what they are doing, and in principle it should be possible to tune the weenies out, but there are SO many of them (and don't get me wrong, there are plenty of weenies writing about Windows and Mac stuff too).
The best quote of all time was something on the lines of
Person A: Rails looks nice but I worry about the maintainability of the code
Person B: (in about March) Huh, nothing to worry about, I started a project in November and the maintainability's fine.

From:

covertmusic.livejournal.com

Yeah, but those are just the same people who were contributing bad libraries to CPAN five years ago, or writing yet another PHP messageboard. Ten years ago they were probably writing very-exploitable C. Which I'd probably do if I were to attempt to write C; I still really see myself as a scientist more than a programmer, and as a result I program as a means to an end, not an end in itself...

Anyway, you can usually spot them by mentioning AJAX (not that I have a problem with the technique, or the libraries - ones like JQuery are getting quite sophisticated and certainly very useful) and standing well back.

From:

Point taken. I thought that the hype curve for Rails was unusually high, though. The Pragmatic Programmers Rails book (1st edition - things are hopefully better now, I don't know whether there is a 2nd edition) basically has a last chapter which amounts to 'well, having read this entire book about this wonderful technology you probably want to deploy your application, well there are 3 ways, none of which really work'. And say what you like about C, Perl or PHP, but K&R / The Camel Book / [pick an early PHP book here] didn't have a 'ha ha, only joking' final chapter.

From:

Though clearly this says more about the Pragmatic Programmers than the weenies. And of course, the PP aren't unique, O'Reilly is a big cheerleader for Open Source But Oh My You Still Have To Pay for Our Books technologies. Although it is interesting to note that they have basically given up on their ETEL (emerging telecoms) stream - the reasons for this would be interesting to find out.

From:

Often the user community and developer community overlap perfectly. One person. ;)

From:

Well, in a sense those ones don't matter. In fact one could argue that one of the good things about open source is that when Fred produces his bit of software, if anyone downloade it and finds it useful, at least they aren't completely shafted when they find it won't run on Windows Wombat and Fred has long since lost interest and moved house and email address.

Clearly, as with anything (music / fiction ) there is the problem of finding the good stuff.

It is not the one developer one user ones that I mind. It is the ones where there are a lot of blowhards on the mailing lists talking about their wonderful plans for version 2.0 which lead one to download it only to discover that essentially it doesn't sodding work. And, of course, they have the perfect right to do this, my objection is primarily an aesthetic one about the tragic waste of human effort and ingenuity.

From:

I want software that's written by someone whose mortgage and livelihood depends on the quality

This is one of the reasons I'm such a skeptic about FOSS in something like the phone business. Phones are a cheap mass market consumer good. You want it to _just work_. You really really don't want to be reflashing your phone too often unless you're paid to do it for a living.

Developing a phone takes a significant man year investment of engineering effort to get it working and ready to ship at a quality a punter will like, once you've paid those engineers you actually find the cost of the software is a small fraction. It's why we're seeing a lot of traction in the market place for Windows Mobile because they make it easy to get a device ready to ship.

A good example of their new approach is the new Palm Treo 500v with the new UX.

From:

aardvark179.livejournal.com

As has been said the baseband software is generally separated out. Software controlled radios are good but the the low level software is part of the FCC licensing so it must not be part of a user hackable OS.

I don't worry about the stability of such an open source phone, they'll get that right (or as right as any other major phone manufacturer does) but I do worry about UI design.

I'm not sure why I'm worrying though, I mean UIQ/SE appear to have completely fucked up the later P series phones as regards UI goes, and there's plenty of other awful examples out there, so it's not like supposedly well organised closed source teams with UI designers can't screw that side of things up just as well.

I do worry that there's an X windows box on openmoko's architecture diagram though.

From:

Ok, my 4 pennies...

Point 1:
So far, based on the projects we've done, there isn't a Linux phone out there yet that matches an RTOS (Real Time Operating System), Symbian or WinMo phone for general quality of finish. One of the core problems here is you, as the user, really don't want to be reflashing your phone ROM too much - it's not something I'd do with something I'd paid for, but I'll let people do it with test hardware I'm leant.

Of the several dozen live phone projects we're working on at the moment, only 1 or 2 are Linux and most of those have ended up "dumped" on the market in China because none of the European networks would accept them quality wise. It's why I'm agnostic about Android, there's been several Linux/Java platforms produced and most of them have flopped because in order to keep them working they have to be locked down to a certain extent to avoid somebody buggering up some of the hard stuff.

Point 2: Split into many little points...
This is a tricky one. As another person mentions, the easiest way to get around this is to have a second processor running the radio stack and have a command line interface to the OS - historically this would be AT commands, although most people are moving to an open API. This is easy to build, and is the approach Apple took, however, you'll find it won't be as reliable as a purpose build phone platform like you'd get from Sony Ericsson or Nokia. It also pushes the price up. From what I've been told, this is also true of the iPhone as an actual phone, but most people don't mind because it's shiny.

Having 2 cores also pushes the prices up and makes power management a nightmare, so what you see are most of the silicon providers pushing for single core where the radio and application processor are on the same chipset. This is harder to integrate and get to work, but the results are worth it. Of course, testing and debugging a phone to the expected quality of a typical user is a huge undertaking and requires access to a lot of really really expensive kit, like Anite or Rhode and Swartz network emmulators and a real Faraday cage. I worry that the open source people won't have access to these million dollar goodies.

Finally, network security and networks are a problem which is why, to a greater or lesser extent all the open source options will either be locked down or come without the network logic which will have to be added as a black box later. Manipulating, for example, the Quality of Service system on a WCDMA network can tear down a local cell and I don't think that the phone companies nor the government will be happy with people being sold a hacking tool for networks.

Even then, quality can be a problem. I know of a trial in 2003 where 20,000 test phones pulled down a major network's Base Stations in the London area.

Android is more likely to be a better bet and work better because Google are at least subsidizing the phones, but I just don't see true Open Source surviving an encounter with the demands of consumer quality mass market products with significant 3rd party dependancies.

From:

dgh.livejournal.com

I suspect that most companies that deploy a free software based stack will lock down basic functionality. Having a phone based on free software doesn't necessarily mean that you can modify the software on it. (GPLv3 takes a stance against this, which means that anything you want to lock down can't use GPLv3 software.)

Motorola have been using Linux on their phones for a while:

http://www.news.com/2100-1001-984424.html

However, this has not translated into any additional freedom for Motorola customers.

From:

It's used as the OS on a fair number of Chinese "feature" phones. They're locked down and won't run applications and they're cheap. However, China is something of a dumping ground for handsets that operators elsewhere won't touch. Mitac were able to ship hundreds of thousands of their Windows Mobile devices there even though they'd been refused by every European and US operator.

From:

teaparty.net (from livejournal.com)

to address the immediate question: yes, this is still very much a work-in-progress. still, take heart from http://informationhunger.blogspot.com/2007/11/openmoko-is-growing.html - his comment of 2007-11-24 that "the dialer and sms handling is also being worked at in a furious pace by the OpenedHand guys. Since about a week, calling has worked without any problems for me." suggests that your fears of 2007-11-13 have been addressed.

note also that the recent discovery of a power-management hardware bug means that the final device has to be respun again; GTA02v5 is now looking to ship in early february. the aim of the game is to have good-quality code by ship-time; if things continue as they have, this is likely to happen.

and now, the wider question. the aim of the game isn't to deliver a slick consumer device. if you want one of those, oh consumer, then buy an iphone, a nokia, or your shiny-phone-of-choice, but for the love of Steve, *stop bloody complaining about how you can't run ${APP} on it, and how the vendor tries to punish you if you do*. the aim of this game is to offer a *choice* to users: if you want a sealed device, with a predictable user experience, then buy one, and live with the user experience you've plumped for; if you want an open device, which will require understanding and love and attention, but is at least as fixable as it is in need of fixing, then go openmoko.

why would i do this? personally, integration of spamassassin and SMS strikes me as a must-have; i am aware of no 'phone that offers anything remotely like this ("this" being a combined bayesian and rules-based inbound SMS filter). even just procmail for inbound SMS would make my life much better. i have no idea if anyone else on the planet wants this, but i do, and i can see how to do it on the openmoko. i expect that others who have strong feelings about what they want their devices to do, will have comparable, yet different, must-have lists. their lists will not impact on my 'phone, unless i really like something that (eg) simon likes on his 'phone, in which case i can put that on my device, too.

andy powell's must-have that he writes about at http://blog.automated.it/2007/11/21/here-i-am-right-here/ , code which can automatically contact his wife as soon as (but not until) he arrives at the Channel Tunnel, is a great example. i have no need for that, and i doubt most people will; but he does, and he can have it, without overloading my 'phone with bloaty code i don't use.

to briefly address daveon's fears about reflashing: i can't comment on openmoko _per se_, as i haven't got one yet, but rockbox (which i run on my ogg player) distinguishes between updates to the boot code (which is capable of allowing a new bootable image to be uploaded, and is itself updated very rarely) and updates to the bootable OS (which does most of what the device is supposed to do, and is updated continually). this tends to defuse most of the scenarios that end up bricking a device; if bleeding-edge doesn't work, upload last-known-good and continue with life until yet another bleeding-edge version is available to try. what i've read about openmoko's cramfs makes me think that they're taking this approach.

From:

Couple o things...

Spam SMS? Do you get these? Seriously, you shouldn't be - SMS is a network protocol and anything being sent is via their kit and they charge for it. So, in theory, you shouldn't get any spam. Filtering on rules is something more interesting and possible with MS and Symbian stuff.

The problem for anybody working in this is that Phones are a shiny consumer product and the addressable market for anything else is going to be pretty small. Making phones work is massively expensive in terms of man hours and I just can't see the model for this. I just can't.

From:

teaparty.net (from livejournal.com)

yes, i get spam SMS. around contract-renewal time, i get a bunch of highly-undesirable get-your-service-through-us messages, which are definitely spam. moreover, i have automated systems sending me SMS, and i'd like to do client-side filtering on those so i'm not alerted except when i want to be. i could do some of that server-side, but (eg) the phone knows where i am, and the server doesn't, which makes client-side a better place to do "what time is it where i am today, and do i like being woken at that time" filtering.

i'm not convinced that "making phones work is massively expensive in terms of man hours and I just can't see the model for this.". making operating systems is massively expensive in terms of man-hours, but these can be and are made for free (see http://www.dwheeler.com/sloc/redhat71-v1/redhat71sloc.html , a 2002 estimate that red hat 7.1 involved around eight man-millenia of effort).

i'm not saying that all complex things can be made successfully on the free-software model. i'm saying that some can; mere complexity is no bar to a free-software model.

i have used open devices in the past that were failures (my agenda PDA, for one), because the code, in the end, didn't shape up. i have used open devices that were very successful (my laptop, under linux; my iRiver ogg player, under rockbox). i don't know which of these openmoko will turn out to be. i hope it will succeed, but i'm aware it may not. i'm not prepared to concede, however, that the project is implicitly impossible.

From:

I'm not saying it's impossible, I just don't see how the NRE costs (outside of the OS) can be handled by anybody sensible. Or rather, I don't see how that can be done and anybody can actually make any money out of it but producing a phone that enough people will want to buy to make it worth the effort. If people want to spend their time doing something for free that will not make them any return then I guess that's their call. I just don't know why, that's all.

As I've said in this thread, it's not just about software. A phone is a blending of Hardware and Software at a very low level. Getting all that to work together well and effectively is really really hard and takes a lot of time and the approval of third parties to agree that you can ship on their networks.

That costs a lot of time and effort and some very real upfront certification costs and typically the use of hardware that costs hundreds of thousands of dollars. This isn't something a group of individuals can generally do, unless they already have access to the equipment, or are prepared to be really generous.

Edited Date: 2007-11-28 07:22 pm (UTC)

From:

teaparty.net (from livejournal.com)

certification's an interesting point. i have no doubt that you're right about it being normal to certify hardware in any given market, but i'm not yet convinced that it's mandatory. if the 'phone's not being sold in this country, by local dealers, yet the device is fully GSM-compliant, is there no middle ground for grey import and unsupported use? i wonder this aloud because i have to do the same thing, all the time, with my internet connections: my connecting firewall is a linux box, which is not certified or supported by any of my ISPs, and if they find me using it, some say they'll cut me off. but because the device is standards-compliant, they can't generally tell i'm using it.

you're also right about a phone being a "blending of Hardware and Software at a very low level", but so is (eg) a music player - and rockbox merrily works on those, without the hardware being designed to support it (in most cases, the hardware has to be reverse-engineered first). so this, too, is no bar in principle.

the idea, as with all open projects, is that the project becomes self-sustaining across multiple combinations: once there's working hardware and working code, the code may be ported to other (more popular) hardware, and other codes may be ported to the hardware. this has been known to happen, and it's also been known, often, to fail.

you may well be right about openmoko failing. hell, it may not even come to market in february. but if it does, the *very, very least* i can do is buy one, and try to use it. if i don't even do that, the project fails, and we're all stuck with closed 'phones that do only what the vendor and the networks will permit until the end of time, i can't complain.

From:

If you want something to attach to a GSM network then you need to pass the basic GCF certification tests and if it is a handset then it will need to have passed SAR testing too. No question. No grey area. It's really not quite the same thing as your home network. If your router or home TCPIP stack is a bit shit it's not that much of an issue except for you. If a mobile phone is a bit shit then it can actually tear down a GSM cell. WCDMA is a lot worse. They'll reasonably quickly spot IMEA numbers and block them at the HLR which will prevent them attaching to a network.

Believe me, they'll do it, as MS found out a few years ago.

I've no issue with the hardware getting reverse engineered, but, at least in mobile phones, the effort to do that runs into tens of thousands of hours effort just to get a BSP loading the OS at sensible quality. I have serious doubts that anybody who isn't being paid is going to be prepared to do it.

To put this in perspetive. The last port to new silicon we did for an open OS took nearly 40,000 man hours of effort.

From:

teaparty.net (from livejournal.com)

you're telling me that you seriously doubt that something that takes 40,000 man-hours can be done for free. i've already pointed out to you that something taking about *sixteen million* man-hours of time was merrily distributed for free (both zero-cost and unencumbered).

i know this may beggar belief, but it's true. i freely accept that doesn't mean that all tasks that take a million man-hours can be done with neither cost nor encumbrance to the end-user, but *some clearly can*.

i don't know whether openmoko is such a task, but that might make a more interesting discussion point than "i seriously doubt that such tasks exist", which isn't hugely convincing.

on the subject of QA, you've sunk a boat i wasn't floating. i happily accept that the networks will block _en banc_ devices which are shown to adversely affect network ops. my question is whether there's a strong correlation between devices which aren't tested on the expensive testbeds you describe and devices which adversely affect network ops.

i note also that the GSM hardware in the openmoko is pretty tightly tied-down on the network-facing side. even the wiki merely notes that "we cannot provide many details on the GSM chipset due to very tight NDAs", but they go on to add that "this is not neccessarily (sic) required, since it interfaces using a standard UART serial line with the S3C2410. On that interface, GSM 07.05, GSM 07.10 and other standardized protocols are used.". i'm no GSM protocol expert, and you sound like you might be, so it'd be interesting to hear your thoughts about that model of interaction with the GSM infrastructure.

WCDMA instabilities are sad but not relevant here, since openmoko don't propose to make a WCDMA 'phone.

From:

i know this may beggar belief, but it's true. i freely accept that doesn't mean that all tasks that take a million man-hours can be done with neither cost nor encumbrance to the end-user, but *some clearly can*.

I suppose it depends on what you actually want to do with the product at the end of the day and if you want to make any money off it as a consumer good. If you just want something that you can play with as a hobbiest and you're ok with hanging around waiting for hard stuff to be fixed or not then sure. But I work in mass market consumer goods which maybe have an 18 month useable life before the person upgrades them. They have to work out of the box. Industrialising a BSP for a new phone is 40,000 hours full time work - 8-10 hours a day, 5-6 days a week for months until it's good enough to ship to the consumers. That's separate to UI work, application integration or anything else. The last large whole phone project we worked on was almost 200,000 hours not included the OS and radio integration work. Doing all that for free and getting something that works? I don't see it.

Then there's stuff that isn't open source and is a royal PITA to work with and get working, like Bluetooth, Java and similar which just eats hours trying to debug.

If the goal is for something people can play with for fun, and somebody is underwriting the hardware and it doesn't really matter if it's a bit flaky, then fine. But we build mass market consumer goods that have to work when they hit the stores, hence we'll probably have 6 months of QA and certification work once we've finished the final build.

We really struggle to find enough people who really understand how a lot of this stuff works to do projects around phones so I'm not really clear on where the people will be coming from to do the hard low level engineering time. Or rather who have enough free time to do it.

my question is whether there's a strong correlation between devices which aren't tested on the expensive testbeds you describe and devices which adversely affect network ops.

In our experience of mobile phone development. The short answer is yes. There's also a strong correlation between the knowledge of the engineers doing the debug and the quality of the device that ships. Skip these steps and, well, you've a HTC phone. Lots of features, looks ok, works like a dog.

For the longest time Microsoft tried to avoid using test beds and expensive equipment - they saw it as something outside of the platform and therefore no their problem. The mental model used was that the radio was basically a wireless modem and therefore wasn't their problem. Consequence, the phones ran like dogs and companies had huge QA and debug overheads. Their solution was to start working with some new partners who took this stuff seriously, and to get in the Anite kit etc... that they didn't have. They did that in late 2005/early 2006. The next generation of software writen that way is _just_ starting to hit the stores now and most of it will be in 2008.

From: