Developed with the Center for Evaluation Innovation this two-part guide to network evaluation includes a brief that outlines the frameworks, approaches and tools to address practical questions about designing and funding network evaluations and aCasebook that provides profiles of nine evaluations.
Download at: www.networkimpact.org/networkevaluation
The civic tech field has expanded so widely in recent years, it’s hard to think of a major city or an area of civic life that these technologies don’t touch. In this dynamic environment, the John S. and James L. Knight Foundation has been a field leader, investing over $25 million since 2010 in projects ranging from neighborhood forums, to civic crowdfunding platforms, to efforts that promote government innovation. For eighteen months, Network Impact worked with Knight Foundation grantees and other civic tech leaders to find out how they measure success, focusing on tools they’re using to track platform performance and assessment challenges they face along the way.
We started by identifying key outcomes related to these common civic tech objectives and gathered case examples of assessments from the field:
- Build place-based social capital
- Increase civic engagement
- Promote deliberative democracy
- Support open governance
- Foster inclusion and diversity
Our work also led us to think about tracking the performance of a platform through its lifecycle – recognizing that assessment priorities vary with stage of development, from early testing of a minimum viable product to later-stage scaling of a tested concept.
The result of this research: two guides to evaluating civic tech that summarize assessment best practices, including leading methodologies and metrics that can help innovators monitor progress towards their goals and evaluate the impact of their efforts. Some of these assessment best practices focus on connections between users, both online and off-line, with an important network dimension.
Assessing Civic Tech: Case Studies and Resources for Tracking Outcomes is a publication of the Knight Foundation with Network Impact that focuses on measuring the impact of civic tech platforms on people, places, and processes.
How To Measure Success: A Practical Guide to Answering Common Civic Tech Assessment Questions is a Network Impact publication that offers examples and advice for monitoring a platform’s ongoing performance using tools and approaches that are effective and practical.
Additionally, the Knight Foundation wrote up their key lessons from investing in civic tech that are also worth a read.
How Code for America is using the Assessing Civic Tech guide
The release of this guide is coming at the right time. Demonstrations of what is possible are up in running in communities of every size across the United States. Now we need to find out not only what works, but what works best over time.
At Code for America, the guide will be particularly helpful for Fellowship teams and volunteer Brigades who are thinking about the questions they need to ask and the changes in attitudes they need to measure to assess progress towards increasing civic engagement and open governance. The process and case studies documented in this guide will be useful for structuring these assessments.
At Code for America, we believe that it is critically important to identify the residents, community groups, or government staff who will be using the particular public service program or benefit, then work with them early in the assessment design process. This guide provides important examples of how to frame an evaluation to include and work with intended beneficiaries. It offers sample questions and resources that will be very helpful to organizations and individuals who are beginning to explore how they can include measures of civic engagement and changing attitudes in their assessment of their efforts.
This final evaluation report describes the progress of the Regional Networks to End Homelessness toward goals set forth by the ICHH such reducing the need for shelter and achieving housing placement outcomes and increasing opportunities for broad-based discussion with diverse stakeholders. Following brief introduction and background sections, the report summarizes the findings of the evaluation in detail; and offers recommendations, based upon these findings for long and short term action. The evaluation informed the United Ways of Massachusetts and the ICHH’s immediate commitment of $1 million to support network coordination in all regions through the following fiscal year, and, as consequence of the pilot results, the state legislature approved Home BASE, a major program that builds on the innovations successfully used in the pilot.
With support from the Knight Foundation through the Knight Community Information Challenge, community foundation leaders and their partners around the country are working to create more robust local information “ecosystems.” It was our privilege to get into the field to see what each of the four community foundations featured in the case studies is doing to promote information healthy communities.
Our eye for ethnographic detail helped to surface some of the real-life stories at the center of these efforts. Here is incontrovertible evidence that accessible, reliable and relevant news and information can enhance civic life and spark community change. Of course, we were particularly alert to the network building dimension in all of this. In addition to the local specifics about news and information, the cases also detail some basic network strategies that are relevant to any social change effort: how to create connections that open information pathways so that people can align and act. Which case most closely maps the challenges you face in your social change work? Any insights here that you might take forward?
The case studies: How four community information projects went from idea to impact can be found here