Investigating the Technology Underlying Top News Organizations

Cross-posted from PBS MediaShift Idea Lab

“We Like Lists Because We Don’t Want to Die” – Umberto Eco

Ever since PBS MediaShift contacted me about contributing to their Idea Lab site again (past posts here), I’ve been taking the opportunity to ponder a new “beat” of sorts, one that would be both interesting to read, and also a contribution in some way to the larger conversation about shifts that are underway in news, media, and technology.

What I’ve decided to try is an experiment: a structured investigation into the technology behind some of the Web operations that are generally considered to be “top news organizations,” something that I’ve been following casually for quite a while. The experiment is to see if I can bring some thoroughness to this line of investigation, and not simply cherry-pick the good technology stories that advance my own views. (It’s easy to discover that I’m a shameless promoter of open-source software and open access.)

This will also be an investigation to explore the who, what, when, where, and why of news operations that are putting their money where their mouth is (or not). For example, do they claim to be in support of user privacy but continue to quietly collect and share users’ information? Do they promote their use of open-source stacks for news application development but run proprietary software to handle day-to-day workflow and content management needs, and – more generally – where do these news organizations fall on a range of technology choices like Web and e-mail service providers, analytics tools, commenting systems, and so on? Hopefully, the answers we find will be informative and useful to other news operations that are facing the same questions.

The first order of business, then, is to come up with an initial set of data to work with. A list of news organizations’ Web sites, if you will, that we can peer into and glean some information.

Building a list (or two)

As a starting point, I looked at lists of top news sites by traffic on the Guardian UK as well as the 2012 report from the Nielsen Group on “Top U.S. Web brands and news Web sites.”

I also scanned a few other lists, which added sites like Mail Online, and then I removed aggregators and sites without much original content, specifically Yahoo News, Google News, Drudge Report, and Reddit.

The final list I plan to work with for “top news sites” is the following:

• CNN Interactive
• The Huffington Post
• BBC News
• New York Times
• FoxNews.com
• The Guardian
• The Times of India
• NBC News
• Forbes Magazine
• The Wall Street Journal Interactive Edition
• Reuters Group PLC
• Washington Post
• USA Today
• ABCNews.com
• Bloomberg.com
• Examiner.com
• Chicago Tribune
• Mail Online

I also thought it would be fun to track some niche lists – here are a couple that came immediately to mind:

The Open News partner news organizations:

• Al Jazeera English
• BBC
• Boston Globe/Boston.com
• Guardian UK
• Zeit Online
• New York Times
• Spiegel Online
• ProPublica
• La Nacion

And a “Semi-comprehensive list of newsrooms doing news applications”:

• Associated Press
• Chicago Tribune
• Cincinnati Enquirer
• CQ-Roll Call Group
• Dallas Morning News
• Los Angeles Times
• New York Times
• ProPublica
• St. Petersburg Times
• The Texas Tribune
• Washington Post
• MSNBC.com

I have a few ideas for other collections of news sites to look at, e.g., Canadian news sites (because I’m a Canuck), online-only news operations (a personal passion), and perhaps the independent newsrooms that make up The Media Consortium membership.

If you have some ideas for other list of news operations and news Web sites to investigate, please drop me a line (Twitter, or the comments here work nicely) or – better yet – just edit the list in Google Spreadsheets directly.

Where we go from here

I’ve asked the folks at BuiltWith – a Web site profiler, competitive analysis, and business intelligence tool – to help provide the basic information for this series. They’ve generously agreed to provide access to their API for the purpose of this column. Using BuiltWith, we’ll have a look at the collections of sites above, explore what technology stacks they are using, and start digging for trends or interesting outliers.

From there, I hope to jump off and dig into the initiatives that aren’t so easily compared such as the technology underlying these news organizations’ mobile and tablet strategies, technical approaches to pay walls, authentication and identity management, and how each organization engages with readers, and so on.

Much of this information is outside the reach of simple profiling tools, so – when I hit those limits – I’ll do something very unnatural for a technically inclined person: I will pick up the phone and start asking questions. All that to say, when the phone rings and it’s me on the other end of the line, I hope you’ll take a moment to field my inquiries.

If there are specific questions that you have for your own news operations, which might be influenced by knowing how other organizations are approaching a specific technical challenge, please drop me a line or leave a comment below. I’d be encouraged to have some reader input on what questions are the most interesting to answer.

Here we go.

Cross-posted from PBS MediaShift Idea Lab. Have comments? Please post them on the original post