But I’m also not happy with the concept of just abandoning the blog, as so many others seem to do. I like the idea of putting it into hibernation, where it still can be linked to and indexed in search engines, but just not active.

So I’ve been working out how to do that. It’s not as easy as I expected. And, yes, worth blogging about…

“Will I Dream, Dave?”

I thought I wanted something quite straightforward. Basically I was going to convert the site into static HTML by walking over it with wget or similar. Then find a host who could serve it up cheaply and reliably.

The cheap part was a requirement but also an expectation. I just thought that someone somewhere would give me some hosting space for static files that would charge about the same as my domain renewal each year. I’m not interested in paying much more than that because, well. I might as well just keep paying the hosting plan I’m on now, and keep the blog open.

I also wanted the new hosting to have cool URIs that don’t change. So if, for example, you’ve bookmarked my insightful not-to-be-missed 2007 post on fixing Ubuntu 7.04 display problems, you’ll be happy to hear that I intended to make sure it serves you the same long-obsolete advice for many decades to come.

And that’s the real trick. As I found out, keeping the URIs the same, is not as easy as it sounds.

wordpress.com

This was the first thing I tried. wordpress.com offers free-to-inexpensive hosting and, hey, I’m already running wordpress, so it should be a snap to switch over. I’d just run the blog as-is with comments disabled and it would be just like in hibernation.

But then I read the fine print, and found that I wasn’t going to be able to keep my current theme with its famous hand-tuned aside formatting. Not ideal, but not a show-stopper either. Also, importing the images and other assets would require involvement from wordpress support. Bit of a pain, but liveable.

So as an experiment I started exporting from my blog and then importing to wordpress.com. How well did this work? Well, about as well as you can expect with software that is not designed to go clunk. Of course, there is no error message to determine what the problem was, nor even an obvious way to erase all the partially-imported content.

Amazon S3

Having resolved by this stage that I was going to convert it to a static site and host it somewhere, I soon came upon the idea of using Amazon’s S3 service. I already had an account, and it looked like a great solution. Cheap, reliable and easy. (Pick any two.)

There are (at least) two problems with using Amazon S3 for this task.

Firstly there is a technical limitation with hosting a “naked” domain (ie girtby.net with no hostname). Basically the way you create a virtual host on S3 is to create a “bucket” with the same name as your host, and then create a CNAME from your domain to that bucketname.s3.amazonaws.com. The problem is that you cannot create a CNAME on the root of a domain, it has to be from a hostname within that domain (eg www.girtby.net will work, girtby.net will not).

So that instantly breaks my URIs, but even if I could solve that problem there’s another limitation. Basically Amazon S3 is not a full hosting environment and doesn’t provide some common web server features. Most notably it won’t serve up / using /index.html.

So I would end up needing my own web server which would perform this redirection. But that defeats the purpose of using Amazon S3; if I had access to a reliable web server, I’d just use that to serve up the site and be done with it.

Google App Engine

Yes, I was quite amazed to discover that you can use Google’s App Engine as a host for an entirely static site. This is an interesting possibility because not only is it a real hosting service, and I can develop software (maybe even a blogging engine!) to bring my site back to life, should I be so inclined.

The plan is almost perfect, but has one flaw. Although Google seem to have once had the ability to host from a naked domain, that ability now seems to have been revoked. Which is a bit unfortunate, as it breaks my URIs.

So that’s really a problem, and one that I must admit I did not forsee when I started using the naked domain as the preferred domain for the blog.

Keen observers will have noted that I have tended to blog each time I try out a new version control system, and this really isn’t an exception. Except that, well I’m not just trying it out, I actually use Bazaar daily at $WORK, so and this is like after-hours practice.

Anyway, I wanted to share this because I’ve found that maintaining a staging and production installation of wordpress, complete with custom modifications and a collection of plugins, is a problem ideally solved by a distributed source control. Plus I really like Bazaar, and wanted to show how easy it can be.

I refer to two different machines here, one of which is the production server (ie my hosting provider), and one of which is the staging server (ie my personal machine). Don’t let the fancy terminology put you off; mentally substitute “my box” and “their box” if it helps you.

Setting up a Local Wordpress Instance

On my staging server I branched the wordpress source from launchpad’s Wordpress repository, which is regularly synced with the official repository:

[s] $ bzr branch lp:wordpress

[I'll use an [s] to denote commands run on a staging server, and [p] for commands run on the production server.]

This command creates a "working tree" of the wordpress source code — a set of files and directories — and an accompanying repository of revisions. At any time the working tree corresponds to one of the revisions in the repository, plus any uncommitted changes. Each commit creates a new revision in the repository. Pretty standard stuff really.

So for a new wordpress installation I add the wp-config.php file and commit it:

[s] $ bzr add wp-config.php
[s] $ bzr commit -m "Added config file"

See the codex for other local setup instructions, I just want to focus on the source control tool for now.

Unless you've used distributed version control systems before, you might be a bit wary at this point, perhaps wondering what happens when I next communicate with the upstream repository. But fear not, this is exactly the point of a DVCS. I've created an independent branch, and the parent branch doesn't even need to know about mine. So, I can quite happily make changes of my own and also merge in upstream changes, knowing that it is all tracked correctly.

But for now let's look at going the other way: publishing my changes to the world.

Uploading to the Production Server

One of the cool things about Bazaar is that it supports many different protocols for publishing branches. So for example, I can just push my branch to the hosting server using sftp:

[s] $ bzr push sftp://girtby.net/home/alastair/wordpress

This will create a repository on the remote server containing all the revisions in my local repository. It will not, however, create an associated working tree. Bazaar does not (yet?) support updating a working tree over sftp. I guess there are too many potential issues with local conflicts and such. Anyway the solution is to ssh into the production box and do a checkout of the published branch:

[p] $ cd ~/wordpress
[p] $ bzr co .

What's that? You don't have Bazaar installed on your hosting provider? No problem - all you need is python. Just extract Bazaar into your home directory somewhere, add the bin directory to your path, and you're away. You don't even need to compile anything.

Of course there are many other tasks to set up a production wordpress, but again let's just focus on getting a Bazaar branch associated with the source files.

Another Way

Of course you could bootstrap everything the other way around. Start with a working wordpress installation on your production server, create a bazaar repository for it, then copy that to your local machine. This would be something like:

[p] $ cd ~/wordpress
[p] $ bzr init .
[p] $ bzr add .
[... bzr rm --keep or bzr ignore the files you don't want ...]
[p] $ bzr commit -m "initial checkin"

You would then branch it locally using:

[s] $ bzr branch sftp://server/path/to/wordpress

Staying in Sync

At this point you have two Bazaar branches, and they can easily be kept in sync as follows. First let's make sure we're running the latest wordpress on our staging server:

[s] $ bzr merge lp:wordpress

This just says to merge the latest changes from upstream. As always with merging there is the possibility of a conflict; you make a change that conflicts with the change on the merge source. In general Bazaar is very good at handling these, and anyway you're very unlikely to encounter them unless you're making modifications to the Wordpress core.

At this stage it's a great idea to test the installation locally. Hypothetically, if there were any unit tests, you'd run them at this point. Otherwise, you can just check that the articles display properly, the admin interface works, and whatever else.

You can even see a summary of the changes that you're merging:

[s] $ bzr status -v
modified:
  wp-admin/admin-ajax.php
  wp-admin/custom-header.php
  [... snip ...]
pending merges:
  westi 2009-02-22 Focus on the first blank field when asking for credentials for upgrade/instal...
    ryan 2009-02-22 Allow editing all of a plugin's files. see #6732
    westi 2009-02-22 Wrap the apply_filters call in a function_exists check as this can be calle...
    [... snip ...]

If everything looks OK, commit and push it up to the production server.

[s] $ bzr commit -m "merge from upstream"
[s] $ bzr push

It should have remembered the push location from last time. Also, just like last time the working tree will need to be updated:

[p] $ bzr up

Eject! Eject!

So I've found wordpress trunk to be fairly stable, but should I ever need it, the DVCS provides a safety net.

If I discover a problem after merging in the latest upstream changes, I can quickly revert simply using the entirely surprising:

[p] $ bzr revert

If, on the other hand I only discover the problem after pushing up to the production server, it's still quite easy:

[p] $ bzr uncommit
[p] $ bzr update

I can propagate that change back to my staging server by merging back:

[s] $ bzr merge sftp://girtby.net/home/alastair/wordpress

But like I said, I haven't had to use this.

Synchronising Media

Changes made on the production server are generally easy to sync back to the staging server - I just download a database dump and import it locally. However media such as images are special because they are not stored in the database. Hence you need a way of getting them back to the staging server. I wonder what the answer could be?

Yep, just commit the changes on the production server and merge them back:

[p] $ bzr add assets/2009/01/funny_picture.jpg
[p] $ bzr commit -m "Added funny picture"

[s] $ bzr merge sftp://girtby.net/home/alastair/wordpress

As noted before, the add command can recursively add all files in a directory, so you don't even need to specify the files individually.

Plugins/Themes

Plugins and themes are handled just like any other change. Just extract the plugin to the relevant directory, and add it to the repository:

[s] $ unzip ~/Downloads/coolplugin.zip
[s] $ bzr add coolplugin

At this point I'd probably activate and test the plugin locally, then commit and push to the production server as before.

Bazaar can also track file renames, deletions and moves, but you obviously have to tell it about them. So when coolplugin is updated, be sure to tell Bazaar about any relevant changes before committing:

[s] $ bzr mv --after coolplugin/oldandbroken.php coolplugin/newhotness.php

The --after switch tells Bazaar not to actually do the move, instead it's already happened and we're just recording the fact.

Why Not...

... Wordpress auto-update? Put simply, I don't trust it. Will it let me manage my own patches to wordpress should they be needed? I don't think so. Also: FTP? What decade is this? Even FTPS, sheesh.

... git? Well no reason particularly. I've dabbled with git, but it never really clicked for me. The concepts and terminology and command set still seem slightly obscure to me: "Want to check in your changes? Just use 'git albatross'! Want to view checkin comments from the most recent merge? No problem, 'git ham-sandwich' is at your fingertips!" OK, I exaggerate a little.

... just shut up already? Oh, OK then.

Of course I have no real idea what I am doing. Despite having zero knowledge of PHP or JavaScript, for some reason I feel no reluctance towards sitting down at a keyboard and bashing away until I produce either Macbeth or a better website. The entire site is stored in a Bazaar repository, and that makes reverting bad changes especially easy, and I’ll attempt to blog further about that sometime.

For now let me just point out a seemingly small change to the site. The dates of the posts and comments are now displayed in a more human-friendly manner such as “3 days ago” or “20 minutes later”. This was a feature of Typo that I liked enough to port to my Mephisto theme, and which (I thought) would be fairly simple to get into Wordpress.

It wasn’t. Read on for the war story.

The Prototype Version

John Wang describes how it is implemented in Typo. Basically it’s a bit of JavaScript that is run when the page loads. It looks for dates, which are delimited by the abbr tag and have a pre-defined CSS class. The script iterates over each of these and converts them into a friendly date. The abbr tag is handy because it allows you to specify the full date and time in the title attribute, which you can see when you hover over the date with your mouse.

One of the challenges with using JavaScript in this way is to ensure that you register event handlers in a manner that works with other JavaScript on the page, as well as across browsers. The Prototype library helps out immensely here, and for actually locating the abbr tags.

One of the nice features of Wordpress is that it manages dependencies on third-party JavaScript libraries, on behalf of the plugins. So as long as your plugin/theme uses the wp_enqueue_script function, Wordpress will find a copy of the requested library and insert your script correctly into the HTML output. You call it simply from the theme’s functions.php file like this:

add_action( 'wp_print_scripts', 'depo_theme_add_javascript' );

function depo_theme_add_javascript() {
    wp_enqueue_script('friendly_dates', get_bloginfo('stylesheet_directory') .
        '/javascript/friendly_dates.js', array('jquery'));
}

So at this point, I had the friendly date script originally from Typo, registered correctly with Wordpress and, it worked. Sort of. Specifically: when I had some of my Wordpress plugins disabled.

Prototype vs jQuery

Besides Prototype there are also other JavaScript libraries that people use, and they don’t always play nicely together. For example, jQuery is quite popular, and seems to be used by at least two of the wordpress plugins I have installed. Which meant that I could use my date script or the plugins, but not both.

Now jQuery (at least) is supposed to play nice with other libraries if you use the noConflict method. Unfortunately I couldn’t get it to work. I even resorted to asking my first question on Stack Overflow. This seemed to stump the experts too; the best answer was basically “switch to jQuery”.

So that’s what I did.

The jQuery Version

jQuery is a quite unusual library. It reminds me a little bit of XSLT, in that you declare a pattern of matching items in the DOM tree, and for each of those, some actions which affect the output. You might even say the similarity is quite ironic, given that XSLT is the browser’s other scripting language.

By chance I came across yet another pretty date JavaScript from Zach Leatherman (but with ancestry tracing back to jQuery author John Resig). This allowed me to perform the article date transformations almost trivially:

jQuery(document).ready(function() {
    jQuery("abbr.published").humane_dates();
});

Nice. But what about the comment dates? These require two dates, both the comment date but also the date of the article to which it belongs. The former is easy enough, but there was no easy DOM path from the comment to its article meta data. If it was XSLT I could have come up with an XPath expression, but jQuery is unfortunately not that powerful (yet?).

So I settled for a bit of a hack. For each comment, I inserted the article date as a hidden field in the HTML as a sibling to the date abbr, where it can be easily retrieved by a jQuery method.

It’s not an ideal solution of course, and part of me wants to (at least) put the article date in a local cache variable where it doesn’t need to looked up all the time. But another part of me just wants to get it working so that I can start the process of blogging about how I got it working…

Publish and Be Damned

Anyway here’s the code. I’ll find a better place for it if I make any other mods, or if there is any interest in it.

Lessons Learned

I’m not even sure there are many here. Perhaps:

• Don’t mix JavaScript libraries.
• Learn jQuery. It’s powerful.
• C++ is easy compared to diagnosing JavaScript bugs (ok maybe not)

The point is that you, as a mere user of the widget, can’t easily inspect its inner workings to determine the quality of engineering and manufacture, and nor do you typically have the expertise to do so. So you need to rely on the information provided by the vendor. Perhaps it plays on national stereotypes, on technobabble (hello, cosmetics industry!), luxurious packaging (hello, Apple!), or many other techniques.

With software — particularly web-hosted software — it is not so easy. An astute observer will examine the interface of the software in question and look for consistency to established standards, prevalence of modes, and so forth. A lay observer will go on aesthetics, and probably not much more.

As the market for software becomes more and more sophisticated, what are the indicators going to be for quality? “Designed in California“? Or are lay people going to look for some more substantive indicators?

I don’t know but I know what I’d really like it to be. And I know what I’ll be looking more and more closely for, in future.

Made With Unit Testing

Once you’ve gotten the unit testing religion there really is no going back. It fundamentally changes — for the better — the way that you write software.

It can also make you a lot more intolerant towards regression bugs. That’s my excuse anyway for my recent dummy spit over wordpress’ broken comments feed. This is exactly the sort of problem that unit testing can catch. So why wasn’t it? And what assurances do I have that it wont happen again? I am not a wordpress developer, and I don’t care to be one either, but as a user I want to have better confidence in the software I use.

By the way, the fact that I was running an unreleased version of wordpress when I encountered the feed validation bug, is completely beside the point. Regularly run unit tests ensure that even when trunk is broken, it doesn’t remain so for very long. As it turned out, the problem that I encountered did make it into the 2.7 release of wordpress, before being addressed. (Although, as far as I’m concerned, the bug still isn’t fixed, because there’s no unit test to prevent it reoccurring).

Even if you don’t know anything about unit testing, you should be able to factor it into a technology decision. Software package A is unit tested, and B isn’t. Which do you pick, if all else is equal?

Other Types Of Testing?

It might be argued that other forms of testing are an acceptable substitute for unit testing. To which I would say: bullshit. Or perhaps I might say “you are mistaken”, in a more professional forum. Unit tests are by definition automated, and hence a lot more foolproof. Automated testing at higher levels of abstraction is of course possible, but from experience it is far less effective. And manual testing is just not fun.

Alternatively, it might be argued that a more comprehensive indicator of quality throughout the development process is an ISO 9000 accreditation. To which I say: you’ve got to be fucking kidding me. (And let’s defer that discussion to another time).

As an experiment, pick your favourite piece of open source software. Go to the website and work out how to run the unit tests. Check out the latest source code from the repository and see if the unit tests run as advertised. If the unit tests are broken, or (worse) missing, maybe you should start looking elsewhere?

As another experiment, pick your favourite piece of closed source software (if you have one). Ask the vendor whether they have a unit test suite, and if so, what percentage of the code does it cover. Again, depending on the response, you might want to think about switching…

… to software that goes “clunk”.

For a while now I have been in awe of Khoi Vinh’s Subtraction blog, with its amazing and minimal black-and-white design. Since first discovering it I had intended to shamelessly copy it, if ever I decided to redesign my own blog.

The original plan was to follow Matt’s recommendation and customise the Sandbox wordpress theme into some reasonable facsimile of Khoi Vinh’s site, without being too noticeably similar. While casting around for other sites to steal CSS from, I came across a wordpress theme which looked to provide most of what I wanted, so I used that instead of Sandbox. Thanks to Derek Powazek for his DePo Clean theme.

My version of the theme has departed slightly (or perhaps radically, depending on your point of view) from the original. Instead of a 5-column fixed layout, I’ve gone with a mostly-fluid layout. Maybe it’s the coder in me, but it just feels wrong hard-coding magic pixel numbers into the stylesheet…

The fluid layout may turn out to be a mistake, and indeed I’m already concerned about the mismatch between the columns in the footer and those in the body. By the time you read this I may have already changed my mind, and reverted to a fixed-width layout.

One addition I am moderately proud of is the linkroll at the bottom, which is sourced directly from delicious via javascript and styled to match the theme of the site.

For the most part everything has come together nicely but I’m afraid I would be lost without the assistance of the awesome CSSEdit, which as you can see, enables a dilettante designer like me to butcher the creations of others with ease.

I have some further creative additions to the site planned, not to mention some actual content, as well. Happy 2009, folks.

Actually, there’s a real chance you might not wonder this. But let’s press on anyway for the sake of Google and future generations.

All the hard work was actually done by Daniel Dave Murphy over at schwuk.com who published the script and a bzr branch. Jason Morrison added MySQL support and published a diff.

I’ve made some changes of my own and published the results. You can get either:

• Just the latest m2wp.py script incorporating Jason’s and my patches, or
• A bzr branch, continuing the spirit of being all modern and distributed.

So what sort of changes did I have to make?

Well, Dave’s script extracted the rendered HTML into the Wordpress database. This is OK but it means you can’t go back and edit old posts in their source form (in my case, Markdown format). So to use my version of the script you need to set up your content filters in Wordpress to match those in Mephisto.

There are a couple of other minor changes, specifically handling NULL as well as empty content (for whatever reason I had some of these fields in my Mephisto database), and a minor formatting problem which prevented the “Read More” links from working properly.

Anyway that’s my version of the script, and now that that’s out of the way I can get back to procrastinating about doing anything with CSS.

Now, the long version.

The story starts about 6 weeks ago when my hosting provider kindly decided to upgrade the server I was hosted on. I had, and still have, a shared hosting account which means that they will do things like this from time to time. Most of the time it’s a good thing.

This time it was a bad thing.

Previously I was running my blogging software (Mephisto) on Rails 2.0.2 on Ruby 1.8.6 on CentOS 4.7. In the server migration, CentOS went to 5.2 and Ruby went to 1.8.7. Rails 2.0.2 doesn’t like Ruby 1.8.7.

During this time almost no one noticed that anything was awry at girtby.net, thanks to the fact that most of the site was being served directly out of the cache that Mephisto keeps/kept for performance reasons. Without any blogging software behind it, the site became a zombie. Minus the staggering gait and hunger for brains of course.

For the next couple of weeks in my copious spare time (which wasn’t much after $WORK and $LIFE) I was trying to get Mephisto to work with Rails 2.2 which could work with Ruby 1.8.7. It soon became apparent that I was about the only person on the internets that was actually trying to do such a thing. And, well, screw that.

So I caved. And after dallying with the Rails floozy for all this time, I went grovelling back to old faithful Wordpress. Fortunately she took me back…

In my ~2.5 year absence Wordpress has addressed few of the gripes that I had about it previously, but it still looks like the best blogging platform available (which includes the surrounding development ecosystem).

There is some more blogging software navel-gazing to come, but that’s enough for now.

Now if you’ll excuse me I have to get back to making this place look respectable…