Tag Archives: networks for social change

Network Evaluation: What’s Different?

An evaluation plan recently prepared by Network Impact shows how assessing a network does–and doesn’t–differ from assessing an organization.

The assignment: evaluate the impact of a loose network of 100s of people around the US–on its members and on other people and organizations.

First step–as with an organization evaluation–is to establish the purpose of the network. But then it’s important to understand the form/structure or “shape” of the network, a matter that veers away from organization evaluation. The shape of a network–the ways in which connections/transactions among members distribute and concentrate–affects the functionality of the network. A network built around “key hubs” may be most effective in spreading ideas rapidly and widely whereas a network built around a dense cluster of connections can facilitate the transfer of complex information and promote peer exchange.
What matters next is to determine what the members hold as value propositions for participating in the network. This, too, diverges from an organization evaluation. Even though an organization’s employees will hold value propositions for their work in the organization (they love the mission of the organization; the organization fits their professional path; they need a job), the types of value propositions will be different from those of people voluntarily associated with a network.

Then, it’s on to what is being transacted by members with each other and the degree to which transactions are leveraged through the network to other members. This sort of analysis could be applied to an organization, to learn more about its culture, and the implicit ways in which work gets done. But with a network, it’s an absolutely necessary part of the evaluation, while in an organization it’s more of a discretionary practice.

When looking at the connections among network members, in other words, it’s essential to ask:
•    How are connections configured?
•    What flows through the connections?
•    What is the strength of the connections (intensity, regularity)?
•    How to the patterns of connection structure, content, intensity, and outcome evolve over time?

Answering these questions, along with those about members’ value propositions, provides the basic data for evaluating the network. Quite a bit of this data and analysis is not what you’d need to evaluate an organization’s impact.

Back to top

Network Mapping: A Bump in the Road

Third in Network Impact’s series about network evaluation.

Monitoring changes in a network’s member-to-member connections is integral to network evaluation, especially when a network’s performance depends on its evolution (e.g., from low levels of connectivity to higher levels of connectivity, conversion of weak links to strong links, etc.). One way to display information about a network’s evolution is to create network maps  We use special mapping software to analyze and visually display the information that we gather about network connections and changes over time. We’ve found that network maps generated in this way reveal patterns that are hard to “see” in the raw data and that are difficult to summarize narratively. (Read more about network structure/shape.)

Network mapping for evaluation purposes can be challenging, however. I was reminded of this recently when I set about mapping ties among homeless service providers in Massachusetts. In this case, pilot efforts to reduce rates of homelessness in the state are being implemented through ten new regional partnerships of many organizations. From the start, our evaluation envisaged the production of ten sets of “before” and “after” regional network maps to demonstrate and compare patterns of network change in relations among the partnering organizations.

We started on the right foot. We added a set of “network connections” questions to an online Network Health Survey that was already in the pipeline (network mapping practice #1: don’t over-survey). We discussed the potential utility of the results with network coordinators – not just the value to the evaluation but also to network members who, we thought, might use the visually compelling network maps to publicize and promote their new ways of working (practice #2: establish salience).  We encouraged coordinators to publicize and promote the mapping project (practice #3: pre-notify and follow up with reminders). But, in the end, we were hampered by a low survey response rate from some networks.

In certain kinds of quantitative research, one can make do with a statistical sample. However network mapping of the kind we do requires close to a 100% response rate. We mapped “before” and “after” connections in 6 of the 10 networks and found some interesting patterns. In the other 4 networks, critical information was missing. Any story told in a graphic based on incomplete data would have been misleading.
What went wrong. We delivered our survey by email which has some advantages: people tend to provide longer open-ended responses to e-mail than to other types of surveys; research shows that responses to e-mail surveys tend to be more candid than responses to mail or phone surveys. In this case, however, many of our intended respondents were “fed up” to start with email and, as service providers, were already “over-surveyed” from other sources. (Turns out the problem is wider. The U.S. population as a whole is over-surveyed; response rates in the U.S. for all types and manner of survey are declining as a result). This is something we will pay closer attention to in the future.

Back to top

Talking Networks for Social Change

Plastrik & Taylor talk with network maven Beth Kanter (and her video camera and blog.)
This blog post is a republished version of one written by Beth Kanter, whose blog on non-profit operations and technology is one of the more respected resources on the subject. You can also watch the video of our discussion – Net Gains Authors talk about Networks

Blog post: Drawing Networks on Napkins with Peter Plastrik and Madeleine Taylor

Last week at the Packard Foundation, I finally had an opportunity to meet Peter Plastrik and Madeleine Taylor  face-to-face.

I first came across their thinking and work on building social networks for social change via the resources section on the Barr Foundation web site back in 2006. Peter Plastrik and Madeleine Taylor co-wrote “Net Gains,” one of the first practical handbooks on building and working in networks for social change.   Whether it is a network of organizations or individuals, this handbook provides a wealth of theory and practice on build, manage, and fine tune a network.

Peter is a president and co-founder of nuPOLIS, the Internet presence of the Innovation Network for Communities (INC), a national non-profit helping to develop and spread scalable innovations that transform the performance of community systems such as education, energy, land use, transportation and workforce development. Madeleine is co-founder and principal of Arbor Consulting Partners, a research and consulting group led by senior social scientists.  We talked a lot about network practices. It was a fantastic opportunity to identify similarities and differences between building networks of organizations as well as individuals – and of course how to weave together the two.
There are many parallels to the use of social networks like Facebook. I was particularly interested in hearing their views on how to ignite a network – how it to get it started.  For those who are working on social networks and looking at how to catalyze their crowds on places like Facebook or Twitter – the advice resonated.   Do you know what the group’s value proposition is?  Do you know what the individual value propositions are? (What’s the pork chop factor?) It’s all about building trust and relationships.  It reminds me of Eugene Eric Kim’s point about networks – everybody is people. Peter and Madeleine describe networks as “platforms for relationships.”  And the goal of those relationships can be learning, collaboration, policy, service delivery, advocacy, mobilizing or action.
Peter is one of those people who likes to draw his ideas and at one point he got up and drew a grid on the whiteboard about the different types of networks and what interventions are needed for success.  Later, I found the chart in Net Gains. We also discussed the whole issue of network evaluation and the difficulty of measuring those relationships versus a specific impact.  Also, the idea of faster tools like social network analysis that give us real time information and the need for someone who is embedded in the network as a real time evaluator.  And, of course, what metrics to use.

Madeleine shared a copy of the network health scorecard, a diagnostic tool that networks can use to reflect on how to improve.  She also discusses it in the video above. During lunch, we discussed the field of network building for social change – what’s needed to build this field?   This is the drawing on the napkin that is described by Peter in the video. Peter and Madeleine raised some interesting questions about the use of social media and support of network’s work in a brief outline and I’ve pulled a couple of questions to chew on:
•    What are the hypotheses about the differences social media can make for achieving a network’s goals – learning goals, policy advocacy goals, innovation goals, and others?
•    What patterns can social media use reveal that provide strategic insight for network?
•    How can social media be used to build high-quality connections, a motivating relationship between members and build trust and reciprocity?

One of the topics we discussed was about the skills and practices of network weavers – whether they are working with networks of organizations or supporting an organization’s network of supporters on Facebook.  As Madeleine points out in the video above, a network weaver is looking at how people are connected and what value they are getting from being connected.  A key skill of the network weaver is to pull out threads and pull people together. As Madeleine notes, “it isn’t about everyone being connected to everybody all the time.”  A big part of the network weaver’s job is pattern recognition and that requires a sort of scanning and watching – that takes time.   I also pointed out that it uses a different part of your brain and there is a need to shift mindsets to get other types of work done. I tend to map my “working the clouds” work in short, time boxed bursts.  I tend to do it when my concentration is at a lower point.   But, when I have to write or blog or think about something, I find more and more that I need to stop being social – not do Twitter, Facebook, or email.  I also need to put classical music on my Ipod and concentrate in a different way.  I’ve also found that I need to do something physical to transition between the two – like take a walk or simply walk around my desk. Peter described an interesting framework for thinking about this use of time:

  • Activities that can be done while doing multiple tasks
  • Activities that require quiet and doing that one task
  • Activities that require several days of concentrating, creative immersion, and laser focus on that task

All in all, a great discussion about networks.

Back to top