this blog is girtby.net

Web Forums Considered Annoying

Specialised web forums are commonplace these days. They cover the entirety of the long tail, and are therefore indispensable for discussing obscure, and not-so-obscure, topics.

These days I wouldn’t consider making any serious purchase without at least a brief consulation of the relevant web forum. Technical problems with just about anything can usually be resolved with a well-crafted search through the appropriate forum. Put simply, the web forum is the basic unit of online community these days.

But despite their ubiquity and obvious utility they remain frustratingly limited in lots of ways. In this post I vent some of these frustrations.

In The Beginning Was …

Well, the command line. But some time after that came the Internet. And some time after that, but before the advent of the web, came Usenet. It still exists and, despite its bad reputation, is actually still worthwhile for … certain types of content.

The problems with Usenet have been apparent for about as long as Usenet itself. In fact, there is an entire chapter of the classic Unix-Haters Handbook devoted to it. For me, the problems stemmed from the fact that there was a single heirarchy of groups which needed to satisfy everyone, and ended up satisfying no one. That there should be one taxonomy for all of human discorse is an obvious falsehood. It probably came about from the single, but distributed, administration structure which underpinned all of Usenet.

For a while there it looked like some of these problems were going to be mitigated through the use of private News servers. This allowed use of the Usenet software and protocols, but allowed anyone to be an administrator, and hence decide on the group hierarchy, rules and policies. I remember frequenting the 3dfx News server to find the latest and greatest information about the Voodoo Rush graphics boards, back in the day.

But for whatever reason these did not take off. Despite commonly-available client software (such as in Outlook Express and in Netscape Navigator), private News servers did not proliferate as much as web-based forums, and so are pretty rare today. At the time I felt that web forums were a step backward, and, I still feel that private News servers are superiour to web forums in many ways, despite the latter enjoying many years of active development.

Set My Data Free

Well, as you might guess my biggest problem is that of Free Data. I have never seen web forum software supporting a data export feature where I could retrieve my own posts. The most widely-deployed forum application, vBulletin does not, as far as I can tell anyway, support any form of automated interface, and is not particularly unusual in this regard.

The best you can do currently is to search for your own posts, and manually extract the contents of each result page. This is quite unfeasible in the general case, and also it is lossy because the rendered HTML rarely matches the raw content of each post.

Usability

The other main problems with web fora — as currently implemented by vBulletin and equivalents, at least — concern usability. For example:

• Searching a web forum is almost aways an incredibly painful experience. The results are often presented as a list of threads, often with no clue as to the context of the search terms within. You generally have to search again within the thread to find out whether or not the original search result is even relevant. Just madness.

• Each web forum seems to implement its own markup for posts. The result is usually a variant of HTML with square brackets instead of angle brackets and a slightly different syntax. The question “why not just use HTML?” comes to mind. The JavaScript editor widgets are generally awful and often very little help in inserting the correct markup.

• And speaking of markup, why are forums themselves generally so HTML heavy? I’m looking at a topic index page right now which is over 100K of HTML, and yet contains only 32 topics (including a half-dozen or so of the obligatory “Sticky” threads). 100K doesn’t sound like a lot, but when you’re navigating back and forth all the time, it can really add up. And as for access from a mobile browser, forget it.

• Annoying subtle differences in basic functions makes it very difficult to go from one forum to another. Do you manually subscribe to topics, or is it automatic when you reply to a thread? And how do you view new replies in subscribed topics? (On one of my regular forums it is under “Preferences”, under most others it’s “UserCP”.) Controls for common tasks are distributed seemingly at random, in accordance with the whim of the site administrator’s preferences (and usually difficult to guess).

Social Dynamics

Almost invariably web forums actively promote a social hierarchy amongst the participants. It is extremely common for forums to display the post count and join date for each member next to each post. This is obviously a benefit for site administrators as it attracts participants, specifically those who care about such metrics.

The post count is a particularly prized visible badge of honour. People seem to go to ridiculous lengths, posting the most banal of comments, all in the name of increasing their post count. MMORPG players chasing ever higher levels exhibit similar behaviour I think, and evidently I’m not the only person to notice the similarity.

Of course all communication online is subject to (sometimes unpredictable) effects by the technology, and you should consult Clay Shirky about those. However I find it mildly frustrating that forums are often so polluted polluted with one-liners, non-sequiturs and other fluff all as a result of affording the participant’s post count such high visibility.

Bring Back NNTP!

What makes all these problems, and a myriad of other little ones I haven’t bothered mentioning, is that they were far less of a problem with the NNTP technology we used before the advent of web forums. And there’s no reason why NNTP couldn’t be reinstated.

In fact there are forums which provide both web and NNTP interfaces. Like Adobe’s for one.

The only downside with NNTP is that there is, as far as I understand anyway, no way to communicate to the server which posts have been read and which are still unread. This really needs to be tackled, but I’m sure the web forum vendors could get together to drive an NNTP extension in this area.

But maybe NNTP is too obsolete to rescue. Maybe the solution is just to kick the forum software vendors to lift their game. Either way I’m happy, actually.

I too find web forums to be very annoying. It is so much harder to find useful information in web forums than it is with Usenet or even mailing lists.

Here’s another idea on how to deal with this problem.

A common format for forums and lists

I am hoping to see Atompub-based forums take off in the future. The HTML forum UI would be a mere façade: a browser-based Atompub client talking to an Atompub service that treats all clients equally.

In terms of technical merits, that would be miles ahead of NNTP… not to mention it’s actually likely to happen.

Good rant, Alastair. I agree with almost every point.

Couple of minor wrinkles:

I happen to post a lot on one forum, BloggingHeads.tv. Before they switched from Phorum to vBulletin, the number of posts was prominently displayed next to one’s uname. Now, it still is, but it’s much less obtrusive.

As it happened, other people used to accuse me of posting just to boost my count, an observation which could not have been more wrong. It got to the point where I wished the post count would go away, just so my number didn’t set other people off. I even considered starting to post with a new uname, just to stop the noise. but I finally decided that since I didn’t care about my number, I shouldn’t care what other people said about it.

Second, regarding BBcode or the like in place of HTML: I disagree with you; I can see the thinking behind it. As annoying as it is to us non-malevolent commenters, it does seem a pretty good way to cut down on security holes and automated comment-spamming tools, while still preserving the desired functionality of markup and linking. Given the state of the Web, I’m happy enough to deal with it, since I do want to be able to, say, italicize, quote, and point to other URLS. I agree with you that it would be nice if there were a more uniform syntax across platforms, but usually the more common elements are the same.

The points where you most resonated with me were the irritations over how crummy forum search tools invariably are and how annoying it is that one can’t automatically retrieve one’s own content. A clenched fist salute is all I can add here.

Brendon, I don’t understand your defence of the custom forum markup. How on earth is [B]..[/B] more resistant to spam and security vulnerabilities than <b>..</b> ?

All they’re doing is making the markup non-standard which is, at best, security-through-obscurity.

I don’t get a lot of spam on this blog due to the obscure blogging platform I’ve chosen. However, I can tell you that of the spam that does make it through the Akismet filter — which is the only anti-spam filter I use, BTW — prettymuch all of it is infested with forum-esque markup such as [URL=blah]..[/URL] and not regular HTML.

I’m not saying that the forums should support the full set of HTML tags. I am saying that for each of the markup functions they choose to support, they should pick the HTML syntax over anything else. It’s simple, it’s common, it’s well-defined, it’s widely-supported.

I’m with you on the HTML bloat problem. Far too little signal-to-noise, substance-to-style, content-to-presentation in many cases that make bandwidth-limited (or -expensive) access painful. A lot of this is a failure to engineer automated HTML generation that separates CSS from XHTML. Ban the frigging <font> tag for a start.

As for the rest, well, I don’t know much about vBulletin et al. I haven’t used an NNTP client for years. Funny, isn’t it? It used to be all the rage back in 1992.

Alastair:

Now that I think about it, you might be right about the HTML vs. BBedit-style tagging. Seems to me I long ago read an explanation of why my claim was true, but I’ll be hanged if I can remember it now. (Uh-oh … senior moment?)