Backward Stability (or: Backstab)

Many many moons ago, when the computing world was young and neckbeards roamed the earth, operating systems were conjured from the ether in a matter of a week or so for the express purpose of playing a video game, using dusty hardware sitting in a dank corner where it had been abandoned by one's employer. In those days, resources were precious, as was time, and people wrote mostly simple software. Over time, sophistication increased, and something we have come to call "paradigm" has emerged as a design concept. Design paradigms include ideas like "everything is an object" and "everything is a file".

Over time, major corporations started figuring out ways to "leverage" (that is, "use as leverage") artificial scarcity and the rare power of throwing mind boggling quantities of money at a problem to produce something that looks almost, but not quite, exactly like a solution. These so-called "solutions" were then sold to business entities with more dollars than sense, and those entities eventually came to be known as "enterprises". Gradually, the most marketable features of a system in enterprise markets resolved themselves into a category of Power Point presentable buzzwords known as "features". It really did not matter what the features were, in the final analysis. All that mattered was that you had the full checklist of features that your competitors had effectively marketed, plus a few more. Old concerns like maintainability and stability got buried under mounds of features.

We are now entering an era where people are really starting to care about some kind of illusion of performance and stability again. To answer the need for an illusion of performance, a new buzzword compliant feature subcategory called "responsiveness" has been invented, and by sacrificing a lot of execution and bandwidth performance we can now write software that often guesses what the user wants incorrectly, but at least does it before the user even uses the mouse to click on some widget, thus providing significant illusion of performance. To answer the need for stability, we simply put some bogus metrics in a full page ad in Businessweek.

This is the corporate software development landscape in which we now live. The guys selling software like it's made up of discrete clumps of atoms that must be manufactured by wizened master craftsmen in some mountain village, or stamped out of an assembly line in Michigan or Japan, are selling features. Because of their aggressive marketing practices and the rampant credulity of middle managers and purchasing agents, these proprietary, feature filled software systems become quite popular. That is now the ultimate aim, and the people doing the software development do not give even half a shit how the software comes out as long as they keep getting their Christmas bonuses, paychecks, and vacation days. As a consequence, software ends up very different from how it looked in the halcyon days of auld where three neckbeards in a janitor's closet recently converted to an "office" of sorts would create a text mode IRC client. Instead, we have tedious spinners and meaningless progress bars and windows that wiggle when we drag them across the screen and CPU meters that look like tachometers reporting that we use 312% of our local CPU capacity, so we still have 107% remaining unused, all attached to something so festooned with button graphics and scrollbars and clumps of emoticon selectors that we have about three square inches of monitor real estate devoted to showing the text of an instant messenger conversation.

Sure, messages don't always get through, but at least we can see them in multicolor Comic Sans with Emoji heart and flag characters and the occasional animated (increasingly misnamed) "smilie" sticking out its tongue and blowing a Bronx cheer at us. That's what really matters.

I will not name any names, but I will describe an all too common situation from my recent real life:

I joined a team of people working on a project. The founder decided to have some videoconference meetings. Most of the people involved didn't care how we had the meeting. A few would have preferred plain text communication; all of us already had access to an IRC channel for the project. A couple thought it might be okay to have a videoconference. The founder really believed that this videoconferencing thing would ensure we had a much better meeting, because much is conveyed by expression. One technology was investigated, and someone couldn't get it to work. Another; same problem, only it was three or four people this time. Another; similarly not working out. Maybe audio would be okay, so several choices were investigated, with better results, but occasionally someone would lose a connection -- even on basic teleconferencing services. During conversations, it was common for people to wonder if they were expected to speak, especially because we sometimes could not make out what someone said when it would have very obviously prompted a response from someone who unfortunately had no idea what was said. Somehow, we muddled through sometimes, though there was usually a period during which IRC was used to communicate with someone while trying to help that person get connected. On one occasion, the meeting ended up just going to IRC after a comedy of connection errors, with declarations of eternal enmity to the problem of not being able to get everyone interoperating on some technological wonder that would solve all the world's problems by allowing us to see each other picking noses while discussing project roadmaps. On at least one occasion, a meeting was cancelled altogether, because (other than IRC -- the simpler, lower bandwidth option) we just couldn't get everyone communicating.

I rather suspect that, if we were developing videoconferencing software with the same approach and aesthetic of those neckbeards in the dusty basement corners of yesteryear, we would have performant, smooth, simple, stable videoconferencing software that worked for everyone. We do not take that approach, though. Even those of us who, like those dinosaurs of software development who didn't even understand the wonders of Extreme Programming, now write software with the idea of using it rather than selling it follow the megacorporate model of piling feature on feature at the expense of basic functionality, performance, portability, and stability, because for some reason we've all bought into the notion that we have to "compete" with the "popular" enterprise software that so dominates the world these days. Meanwhile, a handful of mostly abandoned IRC clients may not offer colors beyond a perversely defined ASCII standard set of sixteen colors that utterly fail to provide a tolerable palette of hues still churn along, connecting easily and stably without any significant issues on a plethora of platforms, showing white text on black screens for the most part, and when we're fighting over how to best waste four hours fighting with videoconferencing software that is Enterprise Ready but not actually usable we can at least use those IRC clients for (roughly) realtime conversation as we try to troubleshoot the installs and configurations for our browsers with WebRTC support.

No, that browser doesn't support WebRTC. You should install this other thing, which doesn't actually have a memory leak, but it does suffer a memory fragmentation issue that can't be fixed without rewriting the core of the application, and it does act suspiciously like a memory leak, but because it's not technically a memory leak in the "forgot to unallocate memory" sense we can make statements with plausible deniability to the effect that there is no such bug, and I forgot what I was saying.

Oh, right, browser choice. We have none, because you should install yet another massive bag of features. Don't bother with that stable, simple software that always works. That's just backwards thinking from the 1970s.

Fun References

from boycott systemd:

systemd0 is a replacement for the sysvinit daemon used in GNU/Linux and Unix systems, originally authored by Lennart Poettering of Red Hat. It represents a monumental increase in complexity, a slap in the face to the Unix philosophy, and its inherent domineering and viral nature turns it into something akin to a "second kernel" that is spreading all across the Linux ecosystem. This site aims to serve as a rundown and a wake-up call to take a stand against the widespread proliferation of systemd, to detail why it is harmful, and to persuade users to reject its use, and especially its ubiquity.

from Given that we aren't going to put any effort into testing, what's the best way to test?:

This is the first video game I've played in about 15 years. I tend to think of games as being pretty reliable, but that's probably because games were much simpler 15 years ago. Calculator doesn't have many bugs, either.

from Gordon Moore: Software is too complex:

"The capability of computers keeps growing and the number of applications running keeps increasing, but the people building the interface keep growing the complexity of that," said Moore. "It's not for lack of effort but the software people are losing ground."

from Happy Birthday Tony Hoare, quoted from C. A. R. Hoare's Turing Award acceptance speech, The Emperor's Old Clothes (PDF download):

At first I hoped that such a technically unsound project would collapse but I soon realized it was doomed to success. Almost anything in software can be implemented, sold, and even used given enough determination. There is nothing a mere scientist can say that will stand against the flood of a hundred million dollars. But there is one quality that cannot be purchased in this way - and that is reliability. The price of reliability is the pursuit of the utmost simplicity. It is a price which the very rich find most hard to pay.

from Ken Thompson, source material unknown:

One of my most productive days was throwing away 1000 lines of code.

from Simplicity Sells:

Part of the problem is, ironically, because the industry has put so much thought into making things easier to use. I'll show you what I mean. This is what the computer interface used to look like, DOS. Over the years it's gotten easier to use. This is the original Mac operating system. Reagan was President. Madonna was still a brunette. And the entire operating system -- this is the good part -- the entire operating system fit in 211k. You couldn't put the Mac OS 10 logo in 211k!

from this is why software sucks:

Whether for serious or for fun, phk and djb have each conjectured a massive all encompassing conspiracy dedicated to maintaining the status quo and preventing the development of secure software. I'm not sure I believe in such a conspiracy, but I am certain that should it exist, it could not possibly hope to be more disruptive. On the one hand we have people claiming the OpenSSL API is too broken to work with (they may have a point); on the other hand we have people demanding that we maintain every last misfeature piece of shit function in that API.

from Why Google Wave failed: Too complicated, no fun:

Wading through the hassles of Wave was like standing in an amusement park-length line, only to find your cubicle waiting for you at the end.