Note: This research project was featured in an article in The New Republic “Escape from Facebookistan” in May 2018.
Front Porch Forum’s (FPF) mission is to help more than 130,000 neighbors across Vermont’s 260,000 households connect and build community in their neighborhoods. Using software they co-created with their technology partner, Toronto-based TWG, FPF hosts free online neighborhood forums that provide members with opportunities to share information, goods and services; promote local businesses and contractors; and engage in discussion on community issues. Through the e-newsletters, neighbors talk about neighborly things: missing pets, households items to borrow or lend, crime, and wild-life sightings. They also talk about opportunities to get involved in their community by volunteering or by engaging in town hall, school board or other community discussions. And they often go offline to meet each other face to face, or attend events.
FPF’s technology has several distinct features:
- FPF forums are not threaded discussions. This means there is no direct back and forth between neighbors in real-time. Instead, neighbors who see an issue raised in one e-newsletter can either email the author off-list, or submit a posting for the next e-newsletter, which builds-in lag time in discussions.
- FPF has a team of Vermont-based, online community managers who review all postings
- Postings are ordered first by FPF’s back-end technology, and then reviewed or re-ordered by online community managers.
FPF’s overarching mission to provide a public service to communities drove them to ask: What impact are FPF postings and discussions having over time? Could technology like FPF help to build social capital in communities? With support from the Robert Wood Johnson Foundation (RWJF), FPF partnered with Network Impact to explore these questions.
Social capital is commonly defined as social connections and the norms of trustworthiness and reciprocity that arise from them. Social capital is considered a predictor of health and well-being, economic development and responsive government (Putnam, 1993; La Porta et al. (1997); Knack and Keefer (1997)). In their research, RWJF highlights the importance of socially connected communities, noting that people who feel attached to their place are “likely to be healthier than those who feel isolated or marginalized…and more inclined to take action to improve our own health and the health of others.”
Efforts to better understand social capital and social networks in place-based settings like neighborhoods often focus on the progression from weak to strong ties in a network over time, as strong ties typically indicate greater trust and connection. Recent research suggests that weak ties formed by short, transactional interactions with other people impact well-being (Sandstrom and Dunn, 2014). Part of FPF’s overarching hypothesis about the impact of their technology is that if neighbors have an easy, friendly, no-cost way to communicate daily, then their perception of their neighborhood and their role in it will become richer. They will pay closer attention to local goings on and begin to get more involved. Then, when trouble or opportunity arrives, this collection of neighborly, conversing, helpful neighbors will respond, whether it’s digging out elderly neighbors after a snowstorm or going after one-time funding to build a community youth center. In this light, small acts of neighborliness take on new meaning. In our research, we hypothesized that participating in or witnessing these small acts create weak ties between neighbors that are nonetheless powerful enough to encourage place attachment, a key correlate of social capital.
We worked with FPF to create a Theory of Action to describe how the exposure to e-newsletters might affect FPF members over time. Below is a simplified version. (To view the full, detailed Theory of Action click here ).
To test elements of the Theory of Action, we designed an online member survey that was sent to all FPF members and completed by over 13,000 members. We then integrated member usage data from FPF’s back-end database, which allowed us to match survey responses with members’ online behavior and engagement data (for example, how many times a member has ever posted, which forum s/he is a member of, whether sh/e is a public official). This allowed us to integrate and analyze both self-reported impact data from the survey and actual usage data on behavior patterns.
Results of our analysis confirmed that FPF is helping to build social capital and that witnessing everyday acts of neighborliness is a powerful driver of both online and offline community engagement.
Self-report data strongly suggest that members are driven to be more engaged with FPF by witnessing other members of their community participate in small acts of neighborliness. Notably, this finding also holds for members who gave lower scores when asked to rate their neighborhood and were less optimistic about their neighborhood’s future, both common correlates of low levels of social capital.
- FPF is having an impact on members regardless of how often, or even if, they post. A positive impact is experienced by all members, even those who participate less and have lower online engagement.
- Across all types of forum communities, discussions of local issues were a top-value generator.
- FPF is likely having more of an impact on offline actions in communities than is currently captured. When asked how often they take action offline in their community as a result of an FPF post, 18% of respondents with low online engagement reported taking offline action as a result of FPF once a month or more (compared to 28% for those who are highly engaged online).
- Top factors for remaining a member in FPF were staying connected to members of the community and staying informed about what is going on locally.
This research provides a jumping off point for digging deeper into how technology can enhance opportunities to build social capital in place. Lessons from this research that could be applied and tested in connection with other efforts to use technology to build community in place include:
- Monitor impact: The field of civic tech has advanced considerably in recent years, and many innovators are moving past standard usage metrics to include outcomes-based research and tracking in their platform and tech monitoring. By conducting research that tracks how and why impact is generated for users and their communities, tech creators are able to maximize that impact by increasing platform engagement and social capital building over time. You can find our publications and resources for evaluating civic tech initiatives here.
- Support frequent small acts of neighborliness – To support place attachment and increased social capital in communities, offer both online and offline opportunities to participate in and witness small acts of connection and kindness.
- Create offline ambassadors – Recruit users who are active online who also report taking actions offline to be ambassadors for both the technology and for community building and engagement. By connecting those engaged members with local initiatives you can further explore ways to support active online and offline engagement. And, local offline ambassadors can reinforce the platform’s impact in the community.
- Use information hierarchy to show users that you are responsive to what they value – People in different communities may value different kinds of information. For example, FPF members who rated their community higher found postings on local crime to be the most valuable, while those who rated their community lower found information on local events to be the most valuable. Differences may reflect other variables as well, such as needs and preferences in rural vs. urban areas. Collect research data that describe the highest value generators for different places, as well as what drives users to engage more, and use that data to highlight information that drives engagement and creates the most value (e.g. putting information about a particular topic at the beginning of the newsletter).
- Hyperlocal is not dead: Many smaller communities lack good local news and information sources. The fact that FPF is by, for and about Vermonters was second highest ranked driver of platform engagement. FPF built a service and a company intent on supporting authentic community interactions and opportunities to share information. In a limited analysis of open-ended responses about why they remain a member of FPF, members testified that FPF was the best source of community information they could get. Especially in rural and small communities, there is an information gap that technology is well-positioned to bridge to keep people informed, connected and aware of opportunities.
- Putnam, Robert with Robert Leonardi and Raffaella Y. Nanetti (1993). Making Democracy Work: Civic Traditions in Modern Italy. Princeton: Princeton University Press.
- La Porta, Rafael; Florencio Lopez-de-Silanes, Andrei Shleifer and Robert W. Vishny (1997). “Trust in Large Organizations.” American Economic Review Papers and Proceedings, 87, 333-38.
- Knack, Stephen and Keefer, Philip (1997). “Does Social Capital Have an Economic Payoff? A Cross-Country Investigation.” Quarterly Journal of Economics, 112(4), 1251-88.
- Sandstrom, G. and Dunn, E. (2014). Social Interactions and Well-Being: The Surprising Power of Weak Ties, Personality and Social Psychology Bulletin.
OpenNews is a growing network of developers and designers who are collaborating on open source technologies that journalism needs to thrive. An OpenNews “hub” supports a core set of network resources including Source, a community site that showcases open technology projects, code convenings that assemble small groups of members to work together on last mile solutions and a fellowship that recruits early- and mid-career technologists who are working outside of journalism and places them in newsrooms. These and other network activities, including an annual gathering SRCCON, aim to attract technologists to the field of journalism while strengthening network connections among journo-technologists so they can increasingly work, learn and solve problems together, within and across journalistic settings.
Our research over 12 months – including an analysis of network participation data describing the engagement of more than 1,100 journo-technologists over 5 years, a survey to 514 participating journo-technologists and social network analysis of member connections – yielded insights into conditions that have contributed to the evolution of this highly generative peer learning and action network. Among them:
- Many doors. Designers and developers can enter the network in many ways with options for participation that match the variety of members’ initial value propositions for becoming involved.
- Transparent discussion of network culture. Network leaders and organizers explicitly promote network principles of openness, generosity and inclusion. This has had an impact. Eighty-eight percent of respondents to our OpenNews survey reported that they advocate more for gender balance and diversity in the tech community because of participation in the OpenNews network. One journo-technologist started a journalist of color Slack community inspired by the ethos of OpenNews activities.
- Flexible base of connections: OpenNews provides a foundation for different “nodes” to light up quickly. This flexible base of connections is ideal for peer learning and collaboration in a rapidly evolving field like journo-technology. Per one technologist we interviewed, “With the constant disruption and change that is characteristic of the industry, capacity to adapt is paramount. This places a high premium on access to diverse people and ideas in real-time with low transaction costs.”
Luz Gomez, Knight Foundation’s director for learning and impact explains “…lessons from the project highlight the strength of networks in connecting people and advancing innovation, while providing journalists with a view into opportunities and gaps as they work to advance newsroom change,”
Because this has implications for the many fellowship and awards programs that funders invest in, we included this important finding in our report to the Knight Foundation:
- OpenNews fellows and fellowship alums serve as network “bridge-spanners.” Their exposure to many parts of the network allows them to connect otherwise isolated network clusters.
(click image to enlarge)
Our survey was co-released with a census of “News Nerds” developed by OpenNews. True to the network’s open and inclusive culture, members have been invited to dig into census results and contribute to the learning agenda and planning for a follow up OpenNews survey next year.
The more information you have about the engagement patterns of network members or users of an online platform, the more tempting it is believe that these data alone can tell you everything you need to know. But, until you explore what type of engagement is valuable and why, and what kind of impact that engagement has on people, organizations and communities, your hypotheses about what actually drives outcomes remain untested.
Organizations often assess their network building efforts or technology interventions (or a combination of the two) to be able to come to more definitive conclusions about what works so that measures and indicators can be adjusted and an organization can learn from its experiences. With new technology that tracks people’s behavior (or even old technology like years of paper attendance records from different types of events) you can integrate actual behavior data on engagement over time with survey and other research to get a more comprehensive picture of how value and impact are created through engagement. You can also compare how engagement and other measures, such as number or type of connections in a social network, relate to impact.
With two recent projects, we were able to integrate engagement data with survey and other data to probe the value of different levels and types of engagement. The results offered insights into how impact was achieved and helped both organizations refine their network engagement strategies.
More Engagement on the Community Commons Means More Impact on Users
The Community Commons provides public access to thousands of meaningful data layers that allow mapping and reporting capabilities for people and organizations to explore community health and policy data interventions and best practices.
- Data Collection and Analysis of Engagement – We worked with the Institute for People, Place and Possibility (IP3), the organization that stewards the Commons, to implement an online system to track user-centric data in a searchable, cloud-based relational database. This provided us with data to establish categories for a ladder of engagement based on engagement with core platform activities, such as building maps and reports, connecting to others, or reading tutorials to build capacity for using data.
- Survey Data Collection on Outcomes – After a year of collecting platform data, we launched a user survey to explore what impact platform use and tool engagement had on users.
- Results – Across key measures, the combined data showed greater impact for users who were more engaged. One of the core hypotheses in the Commons’ Theory of Action was that increased engagement with the platform’s tools would increase users’ knowledge, skills and capacity, a hypotheses that was supported by our research. A sample of the findings from this integrated analysis below.
Different Patterns of Engagement in Mozilla Science Lab Correspond with Different Views on Network Health and Outcomes
The Mozilla Science Lab is a network of researchers, developers, and librarians making research open and accessible and empowering open science leaders through fellowships, mentorships, and project-based learning.
- Data Collection and Analysis of Engagement – In order to build a full database of people who had engaged with Science Lab over the years, we used event records, call attendance records, and GitHub data on code contributions and study group participation to create categories for both the level of engagement and the type of engagement of network members. This allowed us to compare diversity of participation – those people who participated in more than one way – to level of participation – those people who participated a specific number of times — as part of our analysis.
- Survey Data Collection on Outcomes – As part of an existing cross-program survey conducted by the Mozilla Foundation, Mozilla Science participants were asked about their engagement in the networks that the Mozilla Foundation supports. Respondents were asked questions about the network’s health, and how they benefited from their participation in the network.
- Results – We found that an individual’s levels of engagement and diversity of engagement correlated in slightly different ways with their reporting on network health and benefits (see results below for an example). Connecting the dots between patterns of engagement in a network and a range of network outcomes continues to be an important part of how we approach our network evaluation work.
In 2008, Lisa Watson was the executive director of the Downtown Women’s Center (DWC), an organization dedicated to meeting the needs of women on Los Angeles’ Skid Row hoping to overcome poverty and homelessness. That year, Lisa received a Stanton Fellowship to investigate the viability of a co-located social enterprise retail store that would offer workforce training to homeless women and generate revenues for the center. Revenues would be used to subsidize housing and supportive services in the pricey Los Angeles real estate market.
For the past ten years the Durfee Foundation has awarded a select number of Stanton Fellowships to social change leaders in Los Angeles with the aim of fostering innovative solutions to some of the city’s most intractable problems. Lisa’s project became a reality in 2011 with the opening of MADE by DWC, a gift boutique and café that offers organic coffee and food along with one-of-a-kind vintage and contemporary women’s clothing, accessories, household accents, and their signature handMADE product line. One hundred percent of the proceeds support the residents of the Downtown Women’s Center, providing the kind of earned revenue that is a vital component of long-term sustainability for most nonprofits.
Prior to the fellowship, Lisa had met a handful of other Stanton alumni, all in the housing/homelessness space. Over the course of her fellowship, however, she expanded her connections to include Stanton fellows with expertise in urban planning, health, education, the environment, and economic development, as well as contacts in the L.A. Mayor’s Office. The interactions with other fellows significantly affected her project’s design as well as its resulting success. “By bringing together smart people from various disciplines in Los Angeles,” she notes, “problems can be viewed through various prisms rather than through a telescope. Solutions and strategies are developed by looking more richly at the problem from various perspectives and disciplines.”
The Stanton Fellowship provides funds over two years for each fellow to think deeply about a specific challenge related to their work and to tease out solutions that will improve life in Los Angeles. The Durfee Foundation deliberately encourages connecting and knowledge sharing among fellows as a way to foster the cross-fertilization of ideas that might lead to new approaches. Stanton Fellows are intentionally selected to represent a wide-ranging spectrum of issues and sectors, with fellows coming from government and social enterprise as well as nonprofits. Key elements of the program include opening and concluding fellowship retreats that overlap with the next/prior cohort of fellows; quarterly get-togethers hosted by a fellow who provides a tour of the issue they are tackling and includes time for fellows to update the group on their projects; and foundation staff matching fellows with program alumni mentors. In addition, every other year the foundation hosts a retreat to which all alumni of the program as well as current fellows are invited.
Enhanced Peripheral Vision
In order to better understand the network dimension of the program, the Durfee Foundation asked Network Impact to assess the role that ties among Stanton Fellows play in contributing to the program’s goals. To that end, in the fall of 2014 we surveyed current fellows and alumni, and supplemented that work with focus-group interviews and Social Network analysis (SNA) to assess the nature of the connections among fellows over time. What we found has implications for funders who are supporting innovation in the social sector, particularly investors in fellowship or leadership development programs who are curious about the wider impact of these initiatives.
The Strength of Loose Ties
“I can ask any Stanton person for their support, and I have done so. Some I only see once a year and that’s fine….I know who to call if I need something.”
It is not uncommon for fellowship programs to cultivate close, trusting relationships among participants as a way to promote more alignment and coordination among participants. For example, bonds that have been deliberately fostered among community leaders who participate in the Barr Fellows Program in Boston have led to increased collaboration among leading nonprofit organizations in that city. In contrast, a social network analysis of the ties among Stanton fellows reveals a different pattern. In the Stanton case, loose ties among fellows resulting in an exchange of “information that leads to new thinking or framing” are more common (see the maps below). Our mapping also shows that participation in the program increases each fellow’s reach, creating pathways to advice and information from a variety of nonprofit leaders. The majority of connections among fellows are not regular, close, or personal. Instead, the overall effect is one of improved “peripheral vision.” With a view to engaging in transformational activity beyond their own “silos,” nonprofit leaders working on different issues can adopt lessons from other settings and better align their plans of action. Our observations offer a different take on the Strength of Weak Ties thesis put forward by Marc Granovetter. Moreover, research on networks suggests that high levels of trust in networks typically coincide with strong, often personal, bonds among members. In the Stanton network, fellows tend to seek advice from peers as trusted sources, not on the basis of strong personal ties.
Below are maps of connections that “provided information that led to new thinking or framing that has been useful in my work” before the Stanton Fellowship, and after. This type of connection saw a 308 percent increase compared to the intensity of connections before the start of the fellowship period.
A Culture of Trust
“I feel free to pick up the phone and call anybody who was a Stanton Fellow, because it’s sort of a common culture that we all share, and it’s a very different level of trust and access….”
Like other social changemakers, one of the things that Stanton fellows value most highly is their access to trusted information from peers. As in most networks, trust lowers transaction costs for peer exchange; in the case of the Stanton fellowship, this network “glue” is the product of a shared set of understandings that the Durfee Foundation fosters between itself and Stanton fellows and among the fellows themselves. As one fellow explained: “We refer to it as a ‘fellowship’, and that is a particularly relevant word for this experience. I think we all feel very strongly that this is a community that matters deeply and has added immensely to our lives, personally and professionally….Stanton calls are always taken and returned, and there is a warmth and common bond that immediately eases whatever else is happening in the day.”
Reflecting on how this culture of trust has been created and maintained, fellows repeatedly cited the efforts of Durfee Foundation staff to encourage them to take risks. One alumnus of the program explained, “They have created this culture that is incredibly healthy and vibrant and encouraging….They invite you to push the envelope and are not fearful of what other foundations might perceive as failure, which is a healthy environment in which to experiment.”
Equally important is fellows’ confidence in the foundation’s capacity to select candidates whose motivations are aligned with theirs: the desire to transform conditions in Los Angeles for the better and to help the most vulnerable in the city. As one fellow observed, “The staff set the standard for how much we all connect. That’s part of the leadership everyone trusts so much.” Another reflected: “I think my comfort harks back to the rigor and savvy with which the selection takes place. It does say something to me about who is in that group of people, that I can trust them, at least enough to reach out to them.” In other words, the mere fact of being selected as a fellow inspires confidence and trust.
Reflecting on her time as a Stanton Fellow, Lisa describes her experience of getting beyond traditional silos in an environment of loose ties but strong trust. “The whole idea of trying to change a city we all care about is a real connector point. It taught me to love L.A. more, and I now feel more a part of it. I was exposed to all these different things that were totally out of my world and experience. As a result, I am involved and engaged in the city in new ways, with different people than I would have been before I was selected to be a fellow.”
Lisa’s project evolved based on her fertile exchanges with other Stanton Fellows and has continued to grow, with more than fifty of the women involved in the training now gainfully employed, and MADE by DWC products now being sold online and in stores in Los Angeles, providing much-needed funds to advance the full spectrum of DWC programs. Our hope is that others looking to nurture innovative social change efforts will look at how their current activities to connect diverse leaders present opportunities to increase the visibility of their grantees’ efforts and will begin to track the value created through those network connections.
Network Impact is looking forward to 2016! As a team, we’ll be exploring new themes as well as deepening our work in some key practice areas. Here are a few questions we’ll be investigating in the coming year…
In what contexts are online communities most useful to networks?
What are best practices for creating and managing online communities that enhance network connectivity, alignment and action?
For several years we’ve been learning from practitioners about challenges and successes in this area. We’ve also developed some resources for tracking the effects of digital platform use on individuals, their organizations and the communities in which they work. (You can read more about these projects: Civic Tech Assessment Guide and Community Commons.) We are now compiling our top lessons learned and will have a blog post on the topic early this year.
How can funders track and improve the network impact of their efforts to connect people at convenings and retreats as well as in online environments that bring diverse groups into contact with one another?
Over the last year, we’ve fielded an avalanche of questions about social impact networks (of grantees, investees, fellows, awardees) that grantmakers hope to catalyze by creating environments in which people with related interests can connect. Our guest post for Philantopic discusses what we learned from a social network analysis of the Durfee Foundation’s Stanton Fellowship. This year, we’ll be looking at strategies including human-centered design that aim to boost positive network outcomes in this area. We’ll also be exploring frameworks and tools for evaluating these “mass-weaving” efforts focusing on the value they produce for individuals, organizations and at the field level.
As big data, and even “medium data” become ubiquitous, who is successfully leveraging data and technology for social change and how are they doing it?
In our work with Kaiser Permanente and the W. K. Kellogg Foundation, we spoke with dozens of grantmakers, innovators and change agents in multiple sectors to find out how funders can catalyze and spread the use of data and technology to advance social change. What we found is that most organizations, including funders, are struggling to use data and technology well. It will take a concerted effort, and strategic investment, to bring the social sector up to speed. In the coming year, we’ll be looking more closely at network initiatives in this domain such as strategies for connecting technologists to nonprofits, to each other and to each other’s open source projects.
Featured in the summer issue of The Foundation Review “Network Evaluation in Practice: Approaches and Applications” authored in partnership with Julia Coffman, Director or the Center for Evaluation Innovation.
We often hear grantmakers talk about leveraging dollars but rarely get to see year-to-year data showing a funder’s investment over time with the additional resources they attract to an issue. The Garfield Foundation agreed to let us review their investments in the RE-AMP network over the last ten years so we could take a closer look at a specific case. Our analysis shows that even foundations with a modest corpus can leverage their charitable dollars many times over by investing in networks. Our latest blog post on this topic is featured in The Stanford Social Innovation Review .
Developed with the Center for Evaluation Innovation this two-part guide to network evaluation includes a brief that outlines the frameworks, approaches and tools to address practical questions about designing and funding network evaluations and aCasebook that provides profiles of nine evaluations.
Download at: www.networkimpact.org/networkevaluation
Connecting to Change the World builds on an earlier resource that Pete and I developed called Net Gains. This latest collaboration with John Cleveland includes examples and lessons that have emerged from our work with social impact networks over the last decade or so. During that time, we’ve been introduced to many new networks and deepened our work with others. As a consequence, we have a better understanding of what makes some networks highly “generative.” By generative, we mean networks with a renewable collaborative capacity to generate numerous activities simultaneously. These are networks that activate members’ connections on an emergent basis as need and opportunities arise.
Examples in the book include RE AMP – more than 165 nonprofit organizations and foundations in eight Midwestern states working together on climate change and energy policies, Reboot- a network of young Jewish American “cultural creatives” who are exploring and redefining Jewish identity and community in the U.S. and the U.K., ten regional networks of state agencies and nonprofit providers that have organized to end homelessness in Massachusetts, and five regional and two national networks of rural-based organizations that are promoting public policies that benefit rural communities in the U.S. In all of these networks, members have been very deliberate about creating, strengthening and maintaining network ties in order to establish a base of connections from which many activities can arise at the same time or over time. This foundation is the starting point for the progression from connecting to aligning to production or joint action that we also discuss in the book.
Based on their handbook for network builders Net Gains, Madeleine Taylor and Peter Plastrik provide strategies for evaluating the work of nonprofit networks for social change. This article appears in the Harvard Family Project’s Evaluation Exchange Periodical issue XIII, released in 2007 and focuses on the unique characteristics and evolutionary paths of networks and how builders can customize evaluation to account for these factors.