The problem with impact measurement in Civic Tech

Matthew Stempeck
Micah L. Sifry

18 Apr 2018, 2:45 p.m.
Auditorium 3

Most Civic Tech products fail to reach the majority of the public for whom we build.

Matt Stempeck and Micah Sifry, the co-creators of the Civic Tech Field Guide, researched available impact metrics from civic engagement technologies built by and for governments, political campaigns, nonprofits, and startups. The wide variety of metrics offered as proof of impact, and the irregular intervals at which metrics are shared, makes comparison difficult. We are able only to ascertain a rough sense of each entity's scale, and occasionally, impact.

Before being transferred to partner organisations, the Sunlight Foundation's APIs were made available to 28,570 registered developers, who eventually generated an average of 543,535 daily calls for information from an ecosystem of applications. Many other Civic Tech products more closely resemble civic engagement social network Essembly, however, which Newsweek reported had 17,000 registered users before its founders moved on.

Matt reviews these findings and the diverse array of impact metrics offered by practitioners, also discussing the importance of measuring an organisation's outputs in relation to their inputs, such as funding, attention, and staff, to determine a more accurate sense of the value provided.

Reliance on quantitative measures can miss value. A true civic engagement victory may not be reflected in the metrics, while a failure can fulfill all quantitative goals. So, to do a better job sharing what's worked, Matt discusses the benefits of qualitative research to establish narratives, embed vital context, and focus on the actual goal of the technology.

Lastly, one of the core conceits of building technology is that other people will use it as was intended. What is the civic impact of unintended uses of technologies, from the hashtag's power to unite movements to the weaponization of ad targeting utilities to spread disinformation? See research into both positive and negative civic externalities of technology, their potential to eclipse our stated purposes, and how companies are addressing them (or not).

Video

Slides

The problem with impact measurement in Civic Tech (Matt Stempeck and Micah L. Sifry - Civic Hall) from mysociety