Background:

In 2015 we began creating an evaluation framework for understanding more about Public Lab's constituency and how we, as a non-profit, can better serve them. (See: https://publiclab.org/notes/Shannon/03-01-2016/creating-an-evaluation-framework-for-public-lab)

While we are embarking on stage two of creating this framework, with thanks to support from the Fund for Shared Insight's Listen4Good program and the Rita Allen Foundation, we have been able to do in-depth work collecting and analyzing data about one group of constituents we are interested in exploring: those who attend Public Lab in-person events. The idea was that the feedback from post-event surveys would help us understand who is coming to the events, why, what people's experiences are, what we could do better, and how people intend to use what they learned after the event. The feedback from these surveys will help us to shape our events and outreach.

The first round of results for the Public Lab Listen4Good project included 77 respondents who attended Public Lab events between July and December 20th, 2016. Attendees participated in events that ranged from the Regional Barnraising and Annual Barnraising, to local workshops such as mapping and spectrometer builds. Below are some of our major findings:

Major Takeaways

This survey brought to light many positive takeaways. We saw that:

• Overall we had a high NPS score (described in detail below) showing satisfaction with Public Lab. People identified that Public Lab is good at promoting collaboration, is open and inviting and promotes environmental exploration. This, outside of being positive, reinforces that people see Public - Lab as serving major aspects of the mission well.
• A large majority of people responded that Public Lab has made a positive difference in their lives. Respondents also identified that people feel they are treated with respect by people in the Public Lab community.
• Most respondents felt that engaging with Public Lab has helped them to become more involved in environmental science and monitoring, and that Public Lab has helped them with tools that they can use to take action on environmental interests and concerns.
• In terms of the demographics of respondents, while the age balance of the group was fairly even, the gender balance showed there were more female respondents at a rate of 65%-35%. This is an exciting statistics as females are considered a minority in both science and tech communities.
• A correlation arose between people who often engage with Public Lab and also believe that Public Lab has made a large difference in their life. This points to the importance of "depth of engagement." It seems if we create the space for people to engage more often, and in more ways, the better of a difference Public Lab can make in their lives.

While there are many interesting things that can be drawn from the results below, Listen4Good is helping us focus in on what can we improve to make people's event experiences better. This is called "closing the loop," the idea being that we parse out what we can change, address it and then survey again. If all goes well, we should find our second round scores increase. So in thinking about things that we can do differently between now and when we run events in the future, here are several considerations:

• A significant number of people who come to events are people who previously haven't engaged in Public Lab. We need to provide more clarity around what Public Lab is and how people can get involved. It also became evident that we need to provide clarity and an avenue for people to see how they can start their own Public Lab project.
• It was identified that we are good at topic based workshops, which could point to the need for workshops being focused on specific environmental questions and campaigns in the future. Of the respondents who did the survey post-OpenHour, most said that the event needed more structure.
• Also noted by survey participants was the desire to see more outreach done to bring in minority groups to events and to promote Public Lab more broadly.
• While we did well on age and gender demographics, one concern was the identified race distribution of respondents, 79% of respondents identified as white.

Results (the raw data)

Our Net Promoter Score

The Listen4Good program includes 6 core questions, one of which is based on a framework called the "Net Promoter System" (NPS) which has long been used in customer engagement circles. This framework asks the question, on a scale of 1-10: 1 being not at all likely, and 10 being extremely likely, "How likely is it that you would reccomend Public Lab to a friend or family member?" From this question, a score can be determined that can help you compare how your organization, or company, is doing relative to others who provide similar services.

Figruing out our NPS score: people who select anything between a 1-6 are considered detractors, this adds to a negative overall score for the organization. Those who score a 7-8 are considered passive and don't add or take away score, those who score a 9-10 are considered promoters and add positive points to your overall score. The total score subtracts the percent of detractors from the percent of promoters.

Based on round one of the Listen4Good survey, Public Lab's NPS score is 82 (we had 64 promoters, 10 passives and 2 detractors). The total average NPS score for all other non-profits surveyed with Listen4Good was 50 at the time of this write up. While it's interesting to compare where we are on that scale, it should be noted that the other non-profits include many other sectors of service including food banks, job placement organizations and housing assistance programs. Eventually we hope to be able to compare our scores with similar organizations, but currently Public Lab is the only nonprofit in this program in the Listen for Good program that serves the Environmental sector.

What is Public Lab good at: Respondents were asked what Public Lab is good at in an open ended question. The responses were then categorized into groups with the following percentage breakdown:

• 34% collaboration (34 respondents) This included responses such as: connecting people across boundaries, and distributing tasks.
• 31% being open and inviting (23 respondents) This included responses such as: facilitating open and welcoming conversations, and Creating inviting and friendly spaces for public participation.
• 21% environmental exploration (15 respondents) This included responses such as: Engaging people in exploring and understanding their environment.
• 15% data/science (11 respondents) This included responses such as: making scientific observations easily and simply performed by the public.
• 14% focusing on "DIY (do it yourself) (10 respondents) This included responses such as: making tools available, and providing the tools, guidelines and kits.
• 12% focusing on low cost (9 respondents) This included responses such as: bringing good affordable solutions to real problems.
• 8% new learning opportunities (6 respondents) This included responses such as: providing heads-on experiences and explaining technical aspects.
• 8% accessibility (leveling) (6 respondents) This included responses such as: bringing a tool into the public grasp.

What could Public Lab do better?

• 16% outreach (8 respondents) This included responses such as:more visible presence internationally
• 12% accessibility (leveling) (5 respondents) This included responses such as: the middle ground between entry point and expert
• 10% clarity (5 respondents) This included responses such as: be more clear about your mission
• 8% provide more resources (4 respondents) This included responses such as: have a public tutorial on monitoring water quality.
• 6% the website (3 respondents) This included responses such as: It's really hard to find resources

How much of a positive difference has Public Lab made in your life?

Overall, how well has Public Lab met your needs?

How often do members of the Public Lab community treat you with respect?

• Out of all the respondents no one replied with Never, Rarely or Sometimes.
• 14% responded Mostly, and
• 86% responded Always

How often do you interact with Public Lab?

What best describes your reason for attending this event?

Please rate your level of agreement with the following statements:1: Participating in Public Lab helps me become more engaged in environmental science/monitoring.

2: Participating in Public Lab events provides me with tools that help me take action on my own environmental interests and concerns.

How likely do you see yourself engaging in the following activities in the future:

Demographics of respondents

Age:

Gender:

Race or Ethnicity:

• 79% White
• 8% Hispanic
• 4% Asian
• 4% multiracial
• less than 4% other

Comparing groups in the data:

Net Promoter Question:

Exploring Promoters Of the 77 respondents on this survey, 64 respondents were "promoters" which means scored a 9 or a 10 on the first question. These are the people who are most satisfied with Public Lab.

All respondents were asked to rate their level of agreement with the following statements:

1. Participating in Public Lab helps me become more engaged in environmental science/monitoring.
2. Participating in Public Lab events provides me with tools that help me take action on my own environmental interests and concerns.

In the total survey, there were four people who "strongly disagreed" with the first statement and two who strongly disagreed with the second statement. All of these respondents were in the "promoters" group.

Passives Passives are people who scored Public Lab a 7 or an 8 on the first question. They are people who are neither detractors from Public Lab, or promoters of Public Lab. One interesting note is that four of the five people who identified that Public Lab could be more clear about things such as our mission, what Public Lab can offer, and what projects are going on in Public Lab in this category.

Detractors Detractors are people who scored Public Lab below a 7 on the first question. These are people who are least satisfied with Public Lab. We had only two detractors in the total 77 people polled for this survey. Both of these individuals attended an OpenHour event and both identified that Public Lab could do better on the event structure.

Difference Public Lab has made in your life

Of those who responded that Public Lab has made "little" difference in their lives, it was one of the only sorted categories where men (4 respondents, 15%) outnumbered women (3 respondents, 7%).

How often do you engage with Public Lab

Most respondents, 39%, mentioned that they engage with Public Lab less often than once every few months. This is most likely because many people who come to Public Lab events are being introduced to Public Lab for the first time.

Of the respondents who interact with Public Lab a few times a month (16 respondents), 13% were passive about Public Lab (score of 7 or 8 on the first question) while 80% were promoters (score of 9 or 10 on the first question). They also were far more likely to mention that Public Lab is good at collaboration (60% of those respondents vs 34% overall). When this group answered the question "How much of a positive difference has Public Lab made in your life?" 55% responded large and 20% responded extreme. This compared to the results of all respondents where 45% believed Public Lab made a large difference in their life and 11% believe Public Lab had made an extreme difference in their life.

Reason for attending the event:

Respondents who attended the event because the topic interested them (33 respondents)

Respondents in this group believed that "clarity" was something Public Lab could do better (22% versus 10% of overall respondents). Overall this group believed Public Lab had met their needs "very well" (67%) more so than the respondents as a whole (48%). Of all survey participants, there were 5 respondents who said they were not at all likely to develop their own Public Lab project in the future, 4 of them were people who joined the event because the topic interested them.

Respondents who joined the event because they wanted to join an organizing effort or campaign about the issue (11 respondents)

These respondents gave a NPS score of 100, which means on the first question, when it came to whether or not they would refer Public Lab to a friend or family member, all rated the answer at a 9 or a 10. This is compared to the NPS score that Public Lab received overall (82 with responses above).

Overall, 66% of these respondents identified that Public Lab is good at collaboration, whereas overall 34% of respondents of all groups felt this way. They also mentioned environmental exploration (33% vs 21%) and new learning (22% vs 8%) as strengths of the organization, in higher proportions relative to the overall group of respondents. Somewhat ironically, two of the four respondents who strongly disagreed with the statement that "Participating in Public Lab helps me to become more engaged in environmental science and monitoring" were from this group.

In terms of how often this group identified that they interact with Public Lab, most respondents commented that they interact a few times a month (63%) and no one interacted more often than that. In the questions about participating in activities in the future, all respondents in this group selected that they were "somewhat likely," "very likely" or "extremely likely" on all categories.

Gender:

Overall there were 26 people who identified as male in the survey and 47 who identified as female. Of the respondents who identified as male, their overall NPS score was 73 versus the as compared to 85 for women (overall NPS of 82). In the question of "how much of a positive difference has Public Lab made in your life," 49% of women responded large and 15% responded extreme as compared to men who responded with 38% large and 8% extreme.

Female respondents:

Male respondents:

In terms of future participation, men tended to show more interest in engaging in Public Lab. There is a higher percentage of men than women who were "very likely" and "extremely likely" to attend a Public Lab event in the future. Men showed a slightly higher interest in starting a Public Lab project. The only converse to these trends was under the category of "likliness in participating in other environmental justice conversations outside Public Lab." There 50% of women said it was extremely likely that they would do this, whereas only 35% of men reported the same.

What's Next

We are working to incorporate feedback from this survey in outreach, events and post event follow ups to come. We will be starting the second round of the this survey in March 2017.