Users As Programmers

This was originally written in March 2006 for my now-retired SOB weblog. It has been rehabilitated and updated somewhat, and moved here.

Application development differs distinctly from some other types of development, such as OS kernel development, in a number of ways. One in particular is how one defines successfully meeting project goals. When writing an application, one seeks to solve the problems, and meet the needs, of its end users. Server and kernel development, meanwhile, tends to focus more on solving the problems and meeting the needs of interoperability with other software.

In some ways, it's easier to write software to the standards of other software than to the standards of the end user. A key factor in this is the tendency of users, and people who are trying to spec out software requirements for user applications, to misidentify the actual problems and needs the application is meant to address. This sort of problem doesn't exist (to the same degree, at least) for software that only interfaces directly with other software. An API is what it is, no more and no less. Software interoperability is a "how" problem: when you set out to solve the problem, the only thing about which you need to concern yourself is how to accomplish the very clearly defined goal. Sure, I'm oversimplifying things a tad, but in general that's all there is to it.

As pointed out by Justin James in an article entitled Programming "How" versus programming "Why", addressing the needs of a user application is not just about how to accomplish a given set of tasks. It's also about why the user thinks (s)he needs those tasks accomplished. Once the "why" of it is determined, you can investigate the best way to address the problems and needs the application really should be addressing, which can lead to a far better "how" than the initial specification proposed.

This sort of thing is addressed by agile programming techniques when an application is commissioned by users, or by some entity that represents the users' interests to some degree. The close communication between developers and users(' advocate) can help to ensure increased success of the project, where "success" is defined as meeting the needs and solving the problems the application is ultimately meant to address. Yay, success.

Of course, there's not always a(n effective) users' advocate, let alone access to the users themselves, when software is being created. In many cases, a corporation sets out to create a "product", with the intent that by solving some problem that has not already been solved, or at least not as effectively or with as many bells and whistles, it can make boatloads of dough for the board of directors and keep all the company's employees in Cheetos and Pepsi (and mortgage payments and so on). These corporations have developer teams (or, worse yet, departments populated by cubicles) that almost operate in a vacuum, basing designs off ideas that have worked in the past, and off their experience of what features people have tended to like in an application, without really having access to the core problems that must be addressed. As a result, we get things like Microsoft Office, Adobe Pagemaker, and Norton Antivirus, which sorta do what's needed but frustrate the hell out of their end users a nontrivial percentage of the time.

For application development of that sort, there's a better way: make the users into developers. When the people writing code, designing interfaces, and speccing out application architectures are doing so because they have a personal stake in getting an application that will solve their problems, the "why" question gets answered. Putting the developers together in a round table setting to consider the matter of what needs to be solved when the developers are also the target users is a great way to develop software that kicks ass.

Do you know where that sort of thing happens? I'll give you a hint: it doesn't happen in the context of major software vendor corporate offices where proprietary, closed source, commercial applications are developed.

It happens in open source development, when a handful of people get together and say "Woah, nellie, there isn't a high-functioning browser out there that doesn't suck." They look at the situation and decide they can do a far better job of defining the project specs for such a thing than anyone else seems to have done thus far. Then they do it. They don't stop solving the problem until the problem is solved to their satisfaction, and being the target market of end users themselves, that means it gets solved well.

Guess what: there isn't a high-functioning browser out there that doesn't suck to some degree. Some friends and I had agreed on that a few years ago, and we discussed the idea of creating the browser we'd like to have. It should, at minimum, do what Firefox (actually Phoenix initially, then Firebird) originally set out to do: provide a lean, lightweight, secure, stable, high-functioning browser that is extensible rather than bloated. For a long time, previous incarnations of Firefox did a passable job of achieving that end, but the project got too caught up in the "alternative to Internet Explorer" aspect of the problem, sacrificing much that was good about the idea to try to lure IE users away with familiarity and similar core features.

Since the original version of this essay, I have written a number of rants about the ills and evils of Firefox, and others have addressed various problems as well; I do not address all that in any detail here, because that's not the focus of this essay. Suffice to say that the Firefox development team lost the plot a long time ago. I was hopeful that we (these friends and I who dreamed of a better browser) could pick up the original plot and start over, doing it "right" this time. I was perfectly willing to be a bullwhip-wielding harsh taskmaster, keeping feature creep and bloatware sensibilities out of the project, and the other people that seem interested in getting involved were similarly inclined. We needed to start with some kind of list of required functionality for the browser and other project specs so we would have somewhere to start. I provided such a list in the original version of this essay, and edited it as needed when the project goals became more clearly defined for a while. It started out as a wishlist, and did not get much further from there:

Things that are probably extensions:

In discussion, we pondered a number of ideas that would be incorporated into the design. One that I brought up, well before Chromium (i.e. Google Chrome) arrived, was a tab management model where each tab ran in its own process.

Some of those items in the first list might actually have been "default extensions", distributed with the standard web browser software package, rather than actually integrated with the core application. Ultimately, however, the idea of tackling this ambitious project fell by the wayside and the essay joined the ranks of ignored old nothings floating around in the backlog of (now retired) SOB, where it was originally published.

Over time, I got rather addicted to several capabilities provided by Firefox extensions, notably including some security extensions and the vi-like interface of the Pentadactyl extension; if you're still using Firefox for some reason, and appreciate the productivity benefits of a vi-like interface in things like editors, I very strongly recommend it. Eventually, however, the downward spiral of Firefox into unusable bugginess, horrid design, and breakage-prone frenetic (and nonsensical) upgrade cycle drove me to distraction. I was about half a hair's breadth from jumping ship to Chromium despite the fact it would mean abandoning things like HTTPS Everywhere that doesn't leak data on first connection or a vi-like interface that didn't aggravatingly break on increasing numbers of webpages, but something rather lucky (for me) happened: the xxxterm browser entered my life. Within a month of trying it out, I was no longer using Firefox at all.

Eventually, xxxterm changed its name to xombrero, and it is still my primary browser of choice. It has a vi-like interface by default, every bit as good for my purposes as Pentadactyl -- better, even, but not due to any failure on the part of the Pentadactyl developers so much as it is a result of the Firefox platform for which they developed their extension turning to feces. I managed to convince the xombrero developers to add functionality similar to HTTPS Everywhere. It is lighter, leaner, faster, smoother, stabler, and far more securely developed than Firefox, without nagging bugs like the long-standing memory fragmentation problem that plagued me as a Firefox user for so long. I have had to give up a little to make that switch, but not much, and far less than I would have to give up to switch to some different browser. While there are still things I would like to see done differently, most of them are limitations enforced by the available rendering engines out there, and I do not anticipate ever being ready to take on the creation and maintainership of a whole new rendering engine package; xombrero uses WebKit, and that will have to be good enough for now.

The xombrero project itself is maintained by people at Conformal Systems, originally created by OpenBSD geek Marco Peereboom, and developed by a bunch of guys who care very deeply about writing secure software. All xombrero project code is distributed at this time under the terms of the ISC License, one of the friendliest software licenses available. These guys are obviously not just "users as programmers", but committed, skilled programmers, as well, and I find xombrero not just the least bad of a bad lot (web browsers being pretty universally awful applications), but a browser that actually aspires to goodness.

Hopefully it stays that way.