1. Although modernist arrogance preaches us otherwise, many old and supposedly dead technologies are in many respects better and, in particular, more trustworthy, than a bunch of new ones. That is not to say that the older – a relative concept – technologies are always that good either, that they couldn't be improved. Email and mailing lists are a case in point. Snail mail is much more reliable than email these days.¹ Web forums and other such trendy forms of masochism can not even be mentioned in the same sentence with email and mailing lists. But even within electronic media, there's another somewhat dead technology that many lessons could be taken from: Usenet/NNTP.

Usenet has suffered an exodus to otherwise inferior mailing lists largely due to the spam problem, which mailing lists have so far been able to rather effectively circumvent by requiring subscription. But required subscription itself introduces various other problems. Just recently, I only got a message because of a strange peculiarity of Gmane's NNTP interface – a bliss compared to to the pain of mailing list subscription and setting up the crapware known as procmail, in particular when you only want to follow a list and not necessarily post. The NNTP interface appears to display messages attempted to be sent on a mailing list it carries, in its corresponding group, if it has received the message through some list it carries, and the headers of the message carry the other list among its destinations. The message never actually appeared on any of the lists I'm subscribed to, because the author was not subscribed to those lists, requiring subscription for posting. But then another problem is encountered: some lists, although they might not require subscription, have not allowed posting through Gmane, so posting a follow-up isn't always straightforward.

To not require subscription means accepting tonnes of rotting spam on the list, if your list provider doesn't provide semi-decent filters. (There are no good ones.) A partial workaround is offered by doing akin to Gmane in its splendour does by asking confirmation on first post to a list: asking to subscribe when trying to post unsubscribed. (Unfortunately Gmane doesn't actually subscribe you; it just tries to confirm you're there for its own purposes.) Few lists do that – none that I know of – and yet it would be a major duct-tape improvement on a fundamentally flawed system. The problem remains that one couldn't care less for all the other threads on that list

The Usenet architecture is in many senses much better than the ad hoc one of mailing lists, but in other respects its hierarchies are a bit too heavy. It is better suited to cross-posting with newsreaders more sanely posting to all associated groups than MUAs retaining recipient information. NNTP/Usenet also allows sensible migration of posts to more appropriate groups with Followups-To headers. It is a paragon of flexibility compared to the mess of a discussion spread over multiple mailing lists. And the trend, Web forums: you can't speak of flexibility in the same sentence, when you have to register an account on every single forum, visit them all manually to check for new posts, and use a crappy Web browser given their crappy HTML-abuse interfaces. But even with all its flexibility otherwise, NNTP/Usenet isn't well suited to creation of ad hoc discussion groups, thanks to its somewhat rigid hierarchies. But neither is mailing list architecture good for that, and that is the crux of the problem I am trying to get to here.

2. I think the thread should be a more fundamental unit of discussion than the group or list. Groups should merely serve as tags for pointing interested parties to potentially interesting discussions or threads. Groups or tags could likewise be threads with messages in a special format that contain a pointer to the content thread – a bit like “planet pages” in blogosphere. Indeed, the architecture should not only provide better discussion groups: it should provide better blogs as well. There is not a fundamental difference between them, after all. Whether other threads can be posted in a group depends on the group's policies, and might require prior subscription or even moderation. Other possibilities, in particular in implementations where threads would live in distributed storage, would be for groups to simply cryptographically sign the initial message of a thread when accepted in the group.

Furthermore, it should not require setting up complex list servers or so to create a thread or a tag. This thread broadcasting system should be to mailing lists what decentralised version control systems are to centralised ones. In this we actually can learn from something more recent: RSS. RSS itself and typical alternatives do suck, however, because it's Ridiculously Simple Syndication, and isn't suited to infrequent polling unless the history in the feed is long. That again wastes bandwidth in the present format. It should be possible to present the whole history of the feed without it having to be downloaded: feed files should be able to link back to older ones, and perhaps the main feed should just provide an index (of recent items) with the actual content in different files. Then no separate archive would be needed either, in our context of replacing mailing lists. (Third parties might, however, provide extra archives and caches.)

Of course, polling is somewhat wasteful compared to pushing, but it does let one set up an ad hoc discussion thread with, for example, just some Web space and an email address to receive posts on the thread to (either manually forwarded in the thread feed/archives or automatically by suitable headers). Providers of established groups/lists could, of course, also offer this service in relation to those groups, but use of one should not be necessary. Alternatively Freenet-style distributed storage, where things live based on their demand, might be employable. Also, polling/self-hosting effectively avoids the group-spamming problem that an NNTP-style system suffers from unless the hierarchy is rigid and centrally controlled. The spam problem at the thread target address obviously remains. I do not have a silver bullet against spam, but it can be fought not only with typical unreliable spam filters, but also with required responses on a first post, as already mentioned, PGP whitelists, and so on. (Equally obviously the logical conclusion of self-hosting threads is self-hosting postings, which might slightly mitigate the problem, but also perhaps make the system even a bit too distributed.) Finally, polling lets one follow a discussion (that doesn't require technical permissions to do so) without subscribing anywhere. Indeed, even in Usenet only servers push between themselves, and users poll from servers. A somewhat similar result can be achieved with caches. Of course, given as widely available and general “stupid” push system (possibly based on ad hoc registration by infrequently polling) as HTTP and Web space is, pushing could also be supported.

3. Anyway, there are many possible implementation architectures for such a thread-based mailing list and blog improvement, many of which could coexist. More important than the exact architecture are the usability and social aspects. The usability problems with present systems were discussed in the introduction. Of course, improved user agent software would be required to make the proposed system convenient to use. As for the social side, such easily migrating threads might help people find the right forum for their gripes and ideas – by creating one themselves, and having it propagate around, without forcing interested parties to suffer the baggage of other threads in a discussion group, crappy Web interfaces, and making it difficult to post a follow-up. The system could also be useful in other applications. For example, given bug trackers based on the same architecture, upstream and distributors (oh, well, fuck them) could also easily share reports (supposing distributors could be arsed to even report their existence back upstream).

4. Do I actually expect something like this to ever be implemented? No. I'm unlikely to personally be able to implement all my ideas in my lifetime. And the FOSS herd, it rather spends considerable time and effort writing clone software for petty reasons of ideology, than actually doing something new and fixing such problems with existing crap as outlined in this and many other of my postings – too many for me to fix everything myself. So I guess we're stuck at a crappy mailing list architecture until even mailing lists get obsoleted in favour of Web forums or other such “modern” crap, following the general societal trend of dismantling what doesn't completely suck, and not improving what does.

---

¹ While snail mail also is not entirely reliable, and probably a greater percentage of all emails reach their destination than snail mail, a great percentage of that mail is spam, and the percentage of real emails lost in spam filters and other points of failure is probably higher than that of snail mails lost. Unfortunately, although snail spam is a lesser problem than email spam, it exists, and the bad thing is that it is quite difficult to built a snail spam filter against godsuckers who will not abide by anti-advertisement notices above the letterbox.