We Muddle Through

How I stopped worrying and love the mud

We don’t figure out how things work. We muddle through.

Steve Krug

Half a year ago, I was lying in the hammock in Southern Italy, reading the first few pages of Steve Krug’s book “Don’t make me think”. The hammock was hung between two — what seemed to be — ancient olive trees, about a 100 yards from the house where we were staying. It was quiet, which itself is remarkable, given the fact that we had our kids with us. Life was good. When I walked back to the house, a big black snake dissapeared behind a water reservoir. I stayed away from the hammock for the rest of the week.

Anyway, as I said, I was reading Steve Krug’s book, and loving it to bits. And I clearly remember being in the hammock, and reading some of his ‘facts of life’, including the one listed above. It reminded me of Brian Foote’s paper “Big Ball of Mud”, an absolute must-read for anyone in software. In a way, both Brian Foote and Steve Krug come to the same conclusion, although they arrive at that conclusion from totally different directions. And in both cases, mud is involved.

Brian Foote’s conclusion is — in a nutshell — that no matter how hard we try to build the software equivalent of skyscrapers, it almost without an exception comes out as the software equivalent of shanty towns. We keep creating big balls of mud. The bitter conclusion from his talk at OOPSLA was — if I remember well — that, perhaps, we shouldn’t even try to resist it. Perhaps we should give up trying to be build skyscrapers altoghether, since shanty towns are in a way remarkably effective.

Steve Krug concludes that muddling through is one of our fundamental human features. Nobody reads a manual. We first see if it we get how it works, even if that means we go off doing things in a totally inefficient way. So in many cases, we don’t even try to do the right thing, we just try to get something done, in any way that gets the job done.

So how efficient is it?

My son went to high school this year. His schedule changes frequently (every day), and I want to be able to quickly find out if his schedule changed. Like, have it automatically change a calendar, or always have the latest version of his schedule on my desktop. The updated schedule is always available as a web page, but I don’t want to go to a web page and wade through all the information I don’t need.

I run MacOS and building an application that puts something on my Desktop would be a lot of work. It would involve setting up an XCode project, finding out about all of the APIs there are, finding the one I need, learning ObjectiveC, writing what I bet is going to be a lot of code, and then also find out how I would be able to parse the web pag, filter out the things I need, and present it in a way that makes sense. It could take days if not weeks, and would clearly take way more time than I’m willing to spend.

In the end, it would however be super-efficient.

However, it can also be done in a much easier way, that would only require a single command. This would be the output:


And this is the command:

curl -s http://... \
  | xsltproc --html /Users/wilfred/bin/schedule-to-xml.xsl - \
  | 2> /dev/null \
  | w3m -dump -T text/html

How I muddle

The first bit of this command just grabs the HTML page that includes my son’s updated schedule. It’s a page provided by his school. Nothing I need to do there. The -s option makes sure the actual content is all you get.

curl -s http://...

The next bit turns that horrible HTML into just the bit I need. The --html option guarantees that the input is interpreted as HTML.

xsltproc --html /Users/wilfred/bin/schedule-to-xml.xsl -

The XSLT file itself is just a small variation of the identity transformation:

<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" 

  <xsl:output method="html"/>

  <xsl:template match="/">
          <xsl:apply-templates select="//table[*/td[text()='Maandag']]//tr[td[@class='tableHeader']]" mode="copy"/>

  <xsl:template match="td[text()='&amp;nbsp']" mode="copy">

  <xsl:template match="td[@class='tableCellNew' and position() = 5]" mode="copy">
      <xsl:apply-templates select="node()" mode="copy"/>

  <xsl:template match="@*|node()" mode="copy">
      <xsl:apply-templates select="@*|node()" mode="copy"/>


So, now I transformed the input into a HTML document with just the table. However, I want this table to be displayed on my desktop. And I don’t have any tools that allow me to put HTML directly on my desktop. I do however have a tool that supports putting text files on my desktop. The tool that is appropriately called GeekTool.

With GeekTool, you can put the output of commands on your Desktop, and make sure it stays there (and gets refreshed). So I need to turn that HTML document that I generated first into a text version of it. Good thing there are tools like w3m, lynx and elinks. Because they allow you to do just that: take the input from stdin, interpret it as HTML and then dump the text version of it. That’s what is happening in the last line of that command:

w3m -dump -T text/html

Setting it up with GeekTool is a breeze, as long as you remember that your single line command needs to have all of these tools available on its path. So you probably need to stick in an export PATH=... somewhere, since without it, it wasn’t able to find any of these things on my system. (I use Homebrew.)

Muddling prevails

So, it works. Perfectly. And bear in mind:

  1. There are no tests
  2. There is no proper error handling
  3. Garbage in (&nbsp references without the trailing semicolon)
  4. Garbage out; I didn’t break out in sweat to clean up the output
  5. I spend half an hour to put it together
  6. 33 lines of code

Nothing beats working software, especially if you can get it with limited effort. This thing will certainly start behaving in a weird way if the input starts to change, or if it goes offline altogether. However, you can simply patch that. Duct tape is cheap, and certainly a lot cheaper than trying to cover for every possible event beforehand.

Mud rules.

(By the way, you should also check out Brian Foote’s talk on video, available here.)