Identify Your Nonprofit's Data Point of View
Most nonprofits collect data of some kind, either because you have to or because you want to. You might collect outputs and outcomes (e.g. meals served, living wage jobs created), participant demographics (e.g. gender, income, special dietary needs), program enrollment dates, program status (e.g. active, inactive, graduate), time allocations, case notes, and more.
The amount of data you could collect can be overwhelming and time consuming. In a prior post, we talked about balancing data needs and service delivery experience, and suggested that time spent “doing data” is a trade off with time spent “doing services.” In this post, we’ll describe some points of view (POV) that nonprofits can have about their data activities. We’ll also point out some of the strengths and challenges with each point of view.
Why Your Nonprofit's Data POV Matters
Understanding your data POV allows leaders in your nonprofit to have honest conversations about:
the work you do today and whether your data is sufficient
the systems you have in place to support your data POV - this includes your processes, people, and tech
the feasibility of doing something new, such as obtaining grant funding that requires you to collect lots of new data or conduct intense data analysis
By the end of this post, I’d like you to be able to:
pick at least one data collection POV that fits your nonprofit
assess if that’s the appropriate POV for your nonprofit today, or if you’d be better off adopting another.
consider whether it’s the best POV for your organization in 1-2 years
Quick Note: The POVs offered below are not perfect descriptions. They are simplified & generalized, and they are meant to be a starting place for understanding.
Point of View 1: Data Collection for Compliance and Accountability
Data collection for compliance (aka Accountability) is about as enjoyable as giving a cat a bath. Nonprofits with foundation or government support know this all too well. Compliance focused data collection is just part of life in the nonprofit sector. It’s a cost of doing business, rather it’s the cost of getting the money you need to do business. Data collection for compliance can be the least rewarding type of data collection.
When you are in a compliance/accountability frame of mind, you collect only the data that’s required to fulfill your reporting requirements. You can see where each piece of data will ultimately go, each report that you will need to submit, and only collect the data that’s necessary.
Adopting this POV doesn’t make data collection simple or easy, however. Funders often want similar but slightly different things. This funder needs to know monthly income at the start and end of the program, and that funder needs to know hourly wages at the end. Therefore, even a compliance POV can require significant planning and resources to do successfully.
There are plenty of reasons to adopt this POV, including:
Resource limitations or funder restrictions against paying for “overhead”
Concern about data collection activities impacting service quality and/or human connections
Mission focus leads to prioritizing service delivery over data
Lack of skills needed to analyze data and glean insights
Strengths of this POV:
Finite resources are spent delivering services rather than collecting tons of data
Participants may spend less time involved in data collection activities
Some programs are a natural fit for minimal data collection (e.g. a small, volunteer managed food pantry might not benefit from anything more than minimal data)
Challenges of this POV:
We have data that we could learn from - such as how your success rates might by demographics - but don't try to learn from it.
Our beliefs can be wrong (i.e. biased), and we can miss opportunities to change our perspective
We lose opportunities to find longer-term program changes that might improve service quality or program effectiveness over time.
When this is our POV, data tasks can feel like they are only a burden, a means to an end, rather than something that we can learn from for future growth.
Point of View 2: Data Collection to Enhance Service Delivery
I once worked for a nonprofit with a very busy food pantry that provided pre-packaged bags of food. A staff member wanted to modify the bags for people with health conditions that required special diets. Great idea!
However, we didn’t want to record sensitive health information like: “Has Diabetes” or “HIV Positive” in our data system. To make it simple and comfortable for everyone, we added a checkbox that said “Special Diet”. When we saw that box checked in our data system, we knew to simply ask the person about their dietary needs. We didn’t even have to talk about their health, we just had to ask about the food.
That tiny piece of new data gave us enough information to serve people better. It wasn’t tracked for compliance, or for program evaluation, or for fun.
This example shows how we can use data, even a few tiny pieces of data, to enhance real-time service delivery. This POV is driven by our mission, and it uses a tool we already have on hand - data about our participants. This POV gives us the opportunity to find ways that we can collect additional data so that we can provide more appropriate, personalized, and impactful services.
Lots of data falls into this category, and many nonprofits are probably already collecting it, including:
Preferred contact method (text, call, email)
Preferred meeting times (morning, afternoon, weekend)
Preferred meeting locations (if you do off-site meetings)
Mobility restrictions
Childcare needs
Even case notes could fall into this category because they can help you understand the feelings, motivations, and challenges of your participants.
This POV is not focused on creating reports for funders. It can help you adjust your services for each person, and it might even help you find areas for larger changes if you see patterns in the data (e.g. everyone would love weekend workshops).
Strengths of this POV:
You are collecting data that can have an immediate and tangible effect on how well you serve participants.
You are adding new data points as you see that you need them, and they would be helpful for service improvement and decision making.
The payoff to the staff who are collecting the data is clear, not theoretical, and not because it helps you win a grant. Because of this, this POV can be satisfying, lead to faster staff buy-in, and make the work of data collection more enjoyable (well...less soul sucking anyway).
Challenges of this POV:
You are asking for data that might not be relevant to a large percentage of your participants. This takes time and resources.
Risk of bloating your data collection. You can find lots of “helpful little data bits and bobs” and suddenly your 10 question intake form turns into a 50 question invasive procedure.
Over emphasis on “edge cases”. An edge case is a 1 in 1,000 occurrence that you think you have to account for. Working around an edge case makes a simple solution complex. An edge case can mean a simple yes/no checkbox turns into a sliding scale, 2 dropdowns, and a graphing calculator.
NOTE: Edge cases are not unique to this POV, but you are likely to run into lots of these situations if you are adding a lot of “just in case it’s useful” type fields. See more on this topic below.
Point of View 3: Data Collection for Program Improvement
When you adopt the Data Collection for Program Improvement POV, you are thinking about your program’s bundle of services, the timing of services, the target population & who you actually serve, your outreach and communication methods, your theory of change, and more.
Your goal is to understand how the things you do and the people you serve contribute to the success of your program. With this POV, you want to learn where you can make changes to improve your success rates, boost your efficiency, or enhance participant satisfaction. If you’re lucky, you can also gain an understanding of what those changes should be.
In a previous post about demographic analysis, we explained how you can use demographics to understand if your services are equally effective with all groups. That’s one simple example of how your nonprofit might use data for program improvement.
With this POV, it's valuable to understand your theory of change because it can help you narrow your focus on potentially helpful data. If you have an idea (a theory) of how your programs and services contribute to the outcomes you want to achieve, then you can focus on collecting data that is relevant. This POV can also lead you to reexamine or modify your theory of change if your programs aren’t working as expected.
Strengths of this POV:
You’ve adopted a learning and growth mindset, so data collection can be seen as an investment in your mission.
You have the opportunity to learn from the data you are collecting and make decisions that are informed by the data, rather than gut instinct or narrow experiences.
If done well, foundations and individual donors will be impressed by your use of data to improve programs, and they might be more likely to fund your work.
Challenges of this POV:
It’s easy to get drawn into collecting “just in case it’s helpful” data points. “Well what if this is the special thing that makes our program more effective and we don’t know it?”
Some of the data you collect for this POV will not have an immediate impact on services. Staff must buy-in to this POV to ensure the data are consistently collected.
The skills and experience needed to correctly complete analysis and interpret results take time to develop.
A poor interpretation of results or mishandling of data can result in misleading findings, especially with complex issues like “Did service A cause outcome B?”.
Data Collection for Proof
This perspective can be considered 1 step forward from data collection for program improvement. From this perspective, you are hoping to collect and analyze data to “prove” or at least “strongly support” claims like “my program is effective at achieving X” and “X is a really important thing to achieve because it leads to some awesome thing, Y”
When you have a Data Collection for Proof mindset, you are looking for data to provide convincing evidence that your program is effective (i.e. you are good at what you say you are trying to achieve) and that your program is impactful (i.e. the things you achieve shape lives and communities in a meaningful way).
Claim of Effectiveness
Our free 16 week vocational training program places 95% of participants in full-time jobs within 60 days after completion, making us one the most effective known approaches of helping formerly incarcerated individuals become employed. Further, only 25% of the people on our waiting list obtained full-time jobs during that same time frame.
With this claim of effectiveness, we are basically saying “we are really good at doing this.” (so give us money to do more of it!)
Claim of Impact
Less than 5% experience return to incarceration within 2 years of completing our program, and more than 50% have living wage jobs with benefits.
With this claim of impact, we are saying “our work matters because it produces long-term benefits to our participants.” (more money = more benefits = better world)
You’re also aware that “proof” is a very high bar so you need to collect the right data in the right way to be able to make a very strong claim about your program. In the claim of effectiveness above, it’s clear that the program tracks people on their waiting list as well as people enrolled in their program. By tracking people on the waiting list, the program is using a comparison group to enable them to make a very strong claim about effectiveness.
Proof of effectiveness or impact claims are often externally focused. They are often meant to impress others or win additional funding. There can be internal benefits from having strong evidence that you do your work well and it has the intended impacts.
Having a Data Collection for Proof POV is great for organizations than have the resources to support it - to develop an appropriate research design, collect the required data, and to complete the analyses that allow you to draw trustworthy conclusions. This might be out of reach for most smaller organizations, however there are some ways to reduce this burden noted below.
Strengths of this POV:
You can make the strongest possible claims to donors that “your work works” and this will help you win future funding .
Your team can be confident that you are doing the right things to fulfill your mission. This can help you build a strong staff culture and a board of directors that will go to bat for you. (It’s hard to go all-in on work that you’re not sure about, especially if you’re underpaid to do it).
It can help you overcome resistance to data collection by showing its benefits.
Challenges of this POV:
The data you need to “prove” your program is effective goes far beyond “we helped 90 people get jobs last year.”
Collecting this data demands time of your staff and your participants, and getting nearly perfect data that you can rigorously examine takes even more time and resources.
There can be trade-off between a focus on rigorous data collection & conducting your research and simply delivering services aligned with your mission.
Since you the data you need to prove effectiveness can be burdensome to collect, consider these ideas:
Think about collecting this type of data sporadically. Can you “prove” that your job training program is effective and impactful only once every 3 years or every 5 years?
Think about collecting this type of data on a subset (i.e. a random sample) of your participants. Can you collect long-term follow up data with only 10% of your participants, who you selected randomly when they started your program? This approach means that you have one-tenth of the additional effort for staff and participants.
Collect as little data as you can to demonstrate proof.
Find areas of overlap with other data goals - if you’re doing a long term follow-up study anyway, ask about a few other topics of interest. Ask how you can make your services more convenient. Ask how you can improve post-program support. Ask what other services people would need to help them achieve even more of something your program cares about.
Just In Case It’s Useful
So this isn’t exactly a POV, but it could become one if you aren’t careful.
If you’ve ever seen a large dataset that social scientists use to conduct research, “just in case it’s useful” seems like the pervasive mindset. There can be hundreds of pieces of data collected on thousands of people. But, a lot of that data is valuable because curious people want to connect the dots. Researchers can look at the relationships between school sector and learning rates in kindergarten, or the relationship between birth-weight and parenting style thanks to these vast data sets.
However, nonprofits aren’t in the business of building vast datasets for scholarly research. We are building datasets that help us get funding to do our jobs and maybe to do our jobs a little better tomorrow than we did today.
It's okay to do a little “just in case it's useful” data gathering because 3 of our 4 POVs have curiosity built into them. This approach should be balanced with “well that wasn’t useful, let's get rid of it” or we get bloated data, long intake forms, and frustration all around.
What’s Next?
Most organizations have to do data for compliance, and that can just be a starting place or and ending place. Think about your nonprofit’s data point of view. Talk with your colleagues about it. You can have one or more, but you might embrace one more than the others. What do you get out of your POV? What does it cost you? Is all this data stuff really paying off for you? Should you start edging into another POV? With this understanding, you can make the best choices for your organization.
Special Thanks
Just like our prior post on Nonprofit Data Strengths, Challenges, and Obstacles
these ideas where shaped by numerous conversations with nonprofit staff across the country. We won't name names, but thank you to everyone who shared your experience and insight.
Reporting your impact is hard when you’re juggling spreadsheets. countbubble makes it easy so you can focus on your mission.
countbubble is nonprofit data management software simplified. Learn how countbubble can help your nonprofit track and report your outputs and outcomes.
Feel free to email us at contact@countbubble.com to start a conversation, or sign up for email updates.
Founder, CountBubble, LLC
Commentaires