Comments are dead. Long live comments!

Cross-posted from PBS MediaShift Idea Lab

Let’s face it – technically speaking, comments are broken. With few exceptions, they don’t deliver on their potential to be a force for good.

Web-based discussion threads have been part of the Internet experience since the late 1990s. However, the form of user commentary has stayed fairly static, and – more importantly – few solutions have been presented that address the complaints of publishers, commenters, or those of us who actually read comments.

Publishers, for the most part, want software that will stamp out trolls and outsource the policing to the community itself (or, failing that, to Winnipeg). Commenters, on the other hand, want a functional mini-soapbox from which to have their say – preferably something that is easy to log into and has as few limitations as possible (including moderation). The rest of us are left to deal with the overly complicated switches, flashing lights, and rotary knobs that we’re expected to know how to use to dial in to the conversation so it’s just right for our individual liking, not too hot and not too cold.

Thankfully, there is an opportunity today to really innovate. New capabilities in the browser, and emerging standards provide an opportunity to completely rethink the relationship between news users and producers – between those who comment and those who are commented upon – and to demonstrate new forms of user interaction that are atomic, aggregated, augmented, or just plain awesome.

That’s why our next Knight-Mozilla Challenge is for you to come up with a more dynamic space for online discussions. You can submit your idea here, and you could win a trip to Berlin to compete with other innovators – or even win a year-long fellowship in a newsroom.

Publishers’ dirty little secret

The truth is, many news publishers don’t actually think comments are a good thing. Or if publishers won’t go so far as to admit that, they’ll usually agree that the so-called return on investment when enabling comments, discussion and debate on their site is not entirely clear.

Therein lies the biggest tension in the “beyond comment threads” challenge: At the end of the day, those who comment on stories, and those who have their articles commented upon, often have very different views on the topic.

Ask publishers about the purpose of comments and they’ll often speak to the very aspirations of independent journalism and a free press: democratic debate, informed citizens, and free speech. Ask them about the reality of comment threads on their site, and a very different picture is likely to emerge.

On the other end of the spectrum are the people who comment. No doubt, for some, it’s their very comment – or comments, in the case of those who actively comment – that creates the value on a given page, not the editorial. For others, the value is in the conversation that coalesces or unfolds in the context of a given story – but, to ease the minds of publishers, always at a safe distance from the “real content,” usually at the end of a story, or well below the fold.

In between are the rest of us, the people who benefit from the tension between publishers and commenters. We rely on the individuals who choose to comment to add context and clarifications, do extra fact-checking (a skill that’s often a casualty of newsroom cutbacks), and, ultimately, to hold the publisher accountable – publicly – and using the publishers’ own soapbox to do so. At the same time, we rely on publishers and reporters to start the conversation and keep it civil.

No wonder publishers are still asking questions about the value of comments: It takes a lot of work to build a successful online community, and the outcome is not guaranteed to work in their favor.

The Slashdot Era

Sometime in late 1997 or 1998, a bunch of hackers who agreed that commenting was broken (or – at that time – just simply missing) on most news sites decided to take matters into their own hands. Enter the era of Slashdot, an early example of the kind of sites that would begin to separate church from state by disconnecting the discussion from the content being discussed. These sites – with lots of comment and little content in the editorial sense – threw some powerful ideas into the mix: community, identity and karma (or incentives).

Fast-forward to today, more than 10 years later, and not much has changed.

Newer sites, like Hacker News and Reddit, continue in the Slashdot tradition, but don’t break much new ground, nor attempt to innovate on how online discussion is done. At the same time, publishers – realizing the conversation was increasingly happening elsewhere – have improved or re-tooled their commenting systems in the hope of keeping the discussion on their sites. But instead of innovating, they’ve simply imitated, and little real progress has been made.

In an era where Huffington Post is the “state-of-the-art” for online discussion, I ask myself: What went wrong?

Enter the innovators’ dilemma

Meanwhile, as the events above unfolded, the rest of the web went on innovating. As publishers and comment-driven communities lamented their situation and pondered how to improve it, the conversation left those sites entirely. The people formerly known as the audience were suddenly empowered to have their say almost anywhere, via micro-blogs, status updates, and social networks.

It was the classic innovators’ dilemma at work. While focusing on how to make commenting systems better, many people didn’t see the real innovation happening: Everyone on the Internet was given their own, personal commenting system. Services like Twitter and Identi.ca solved the most pressing issue for commenters: autonomy. Services such as Facebook and LinkedIn addressed another problem: identity.

Unfortunately, not all innovation is good. Local improvements do not always equal systemwide benefits. That is the situation we are left with today: Comments, discussion and identity are scattered all over the web. Even worse, the majority of what we as individuals have to say online is locked in competing, often commercial, prisons – or “corporate blogging silos” – and is completely disconnected from our online identity.

The Sixth Estate

The opportunity in the beyond comment threads challenge is to radically re-imagine how we, the users, relate to the people producing news, and to each other. It’s time to get out of a 10-year-old box and completely rethink the current social and technical aspects of online discussion and debate. It’s time to stop thinking about faster horses, and start thinking about cars (or jetpacks!).

To get specific, let’s start with a list of great experiences that are made possible with comments:

• Providing value to the publisher: Think about the times that comments have revealed new facts, uncovered sources, or pointed out easily correctable errors. This exemplifies the opportunity for a community to provide value back to a publisher, and helps answer the return-on-investment question. Recently, during the uprising in the Middle East and the earthquake in Japan – when several news organizations introduced real-time streams that mixed editorial content with user-submitted comments – we witnessed a glimmer of something new. What does it look like to push those ideas to their extremes?

• Publishers and users working together: Sites like Stack Overflow (and the other sites in that network) introduced a new standard for directed conversations. More than just question-and-answer forums, these sites attempted to leverage the sense of community on sites like Slashdot and Hacker News, but also direct that energy toward a socially useful outcome, such as collective wisdom. If the Press is a “key social institution that helps us understand what’s going on in the world around us,” then we are all responsible for making it better – reporters, publishers and news readers. So what does that collaboration and the goal of collectively assembled wisdom (other than Wikipedia, of course) look like?

• Holding publishers, or authors, accountable: If the publishers’ aim is to stamp out trolls, the commenters’ equivalent goal is to squelch bad reporting. Many readers expect news stories to be factually accurate, fair and balanced, and free of hidden agendas or unstated personal opinions. Comments were the first opportunity to quickly point out shortcomings in a story (versus a letter to the editor that may or may not be printed some days or weeks later). Think of that span – an immediate retort versus an edited response published well after the fact – and project it into the future, and then ask yourself, “How far could an idea like MediaBugz go?”

The last example on my list has to do with providing value to the community and learning together. How do we address the myriad concerns on both sides of the fence and come out the other end with something that isn’t broken? How can the historical tension between the need for anonymity and the perceived advantages of a real identity be overcome using our knowledge and the tools of the open web? In what way can the visual language of online discussion be taken beyond “thumbs up” or “thumbs down?” And what does it look like to enable commenting on the HTML5 web, which is increasingly driven by video, audio, animations and interactivity?

In those rare inspirational moments – when two sides of a conversation come together and actually listen – there is the nucleus of the idea that inspired the world to embrace comments in the first place. How do we weave that idea into the web of tomorrow? How do we turn up the volume on everything we love about comments, discussion and debate online, without losing what we love in the process?

That, if you accept it, is your mission.

Cross-posted from PBS MediaShift Idea Lab. Feel free to comment here, or on the original.