Here’s an opportunity to make progress on misinformation: track the indicators.

To make progress on big, complex problems, I believe it’s necessary to track signs of the problem, pay attention to the trends not the isolated moments, and apply interventions that consistently make things better over time.

This is difficult unless there’s an organization that’s going to be around for a while, and one that’s adequately resourced to do that tracking. In the case of big problems, like eradicating the impacts of poverty around the world, we’ve got organizations like the World Health Organization and IMF – each tracking indicators that can be used to see the trends. For environmental problems, year-over-year tracking is available from NASA, United Nations, and several others.

For the growing and urgent problem of online misinformation, I believe we need similar, publicly-visible datasets that provide views into where things stand and which way they’re heading. In fact, we’ll probably need many of these datasets, and several organizations committed to tracking them. And I believe that Mozilla, as part of their Internet Health initiative, is well-positioned to be doing some of this work. ¹

Which indicators can measure the impact of fake news?

Let’s dig into this a bit more: I’ll propose that for us to know that progress is being made toward disrupting the creation, proliferation, consumption of misinformation it will be critical to have indicators. These indicators would help track the size of the problem, as well as providing trends over time that can help to explain which interventions are working, and how well they’re working – whether that’s financially-motivated “fake news,” ideologically-motivated disinformation, or rumours and hate speech masquerading as factual information.

The question, however, is: Which are the best indicators to track?

Also, the question of indicators gets a bit more complicated when considering types of misinformation that are not in the category of “news.” Take, for example, false information about health products, misleading financial information, fake reviews for products, and false “facts” published by politically-motivated sources – all of which make their way into search results, circulate in social feeds, and risk misinforming the reader.

Given the above, I’m grappling with the question of which indicators that would give us a view into each of these problems? Or what are the impacts of these problems that are visible “on the surface” of the Internet? For example, I’m wondering:

• How could datasets, like URL database maintained by MetaCert, be combined into a public “State of Misinformation on the Internet” dashboard?

• What surveys of Internet users have been conducted recently, or are going to be conducted, that will surface how individuals are impacted by misinformation?

• Is there insightful research, like recent studies looking at how well students are able to discern content from advertising, that needs to be tracked over time or expanded to other regions?

If you want to change something, track it

To summarize, I’m confidant that we need to be tracking the impact of misinformation in a consistent and regular way, and doing that publicly. I think that many organizations could be doing this for indicators that they’re obsessed with, for example, BuzzFeed News’ list of websites that consistently peddle the worst misinformation, or the bias data that is collected by AllSides.com.

It’s likely that organizations like Data&Society have indicators from their recent report and ongoing research? And maybe your organization has data that you’re tracking that provides a view into the problem? If this data was pulled together, we might just have a set of data that would help us to triangulate and track the size of the problem.

I believe Mozilla’s in a good position to do some of this aggregation work, party because that’s in crosshairs of my fellowship. But mostly I believe that’s true because the Internet Health Report has helped me to think of the misinformation problem as something much larger than just the parts which impact journalism and news. And yet I don’t believe that any one organization is going to be able to make significant progress on its own – this is a massive problem that requires massive collaboration.

So I’ll propose that we’ll need to work together to effectively identify the best indicators to track, because the problem of misinformation is different across communities, countries, and cultures.

Here’s are a few ways we can do that:

• Mozilla is currently accepting proposals for indicators of Internet Health. Submissions are open until June 16. I’ve submitted one or two ideas, and welcome your input on those. If we can get indicators that focus on misinformation into Mozilla’s Internet Health report, they’ll be publically available and tracked over time.

• You can submit indicators that you believe are important indicators to track that highlight the impact of misinformation. You can explain why they’re important, and what data might be available, or needed, to unearth those indicators.

• And you can also let me know personally who the other organizations are that would be well-positioned to take some of this on. Is your organization tracking indicators around fake news, disinformation, propaganda, rumours, or other aspects of misinformation? If so, please shout loudly.

Thanks for reading this far. If you’re passionate about making progress on solutions to misinformation, please say hello on Twitter or LinkedIn so that I can keep you updated on upcoming events, the latest research, Mozilla’s work in the space and more.