Analyze Your Nonprofit Program Data to Understand your Impact
Your social service nonprofit will collect lots of data about your participants (aka clients, consumers, guests) and services. Some of that will be demographic data about your participants, and some of it will focus on your outputs and outcomes to track and demonstrate your performance.
However, having a bunch of data in a bunch of spreadsheets, or stuck in your case management software, isn’t very satisfying. You’ll need to analyze your data to get a payoff from spending your time collecting it.
In this post, we explore a few different ways to examine your data, and we'll connect the dots with a lot of topics we've already discussed in our Nonprofit Data Bootcamp.
Measures of Central Tendency & Levels of Measurement: A Quick Review
Let’s do a quick review of levels of measurement and the measures of central tendency that can be used with each level of measurement.
Measures of Central Tendency
The measure of central tendency attempts describe your entire dataset with a single value. There are three measures of central tendency.
Mean – the mathematical average that you learned in elementary school.
Median – the middle value when all values in a set are rank ordered
Mode – the most common value in a set of values
Levels of Measurement
There are three levels of measurement that you should know: nominal, ordinal, and interval/ratio.
Nominal
Nominal Data is categorical data. The categories themselves don’t have an underlying numerical value and they cannot be rank ordered. No category is inherently “better” or “more” than the others. They are simply different.
Some examples nominal data:
Yes/No
Red/Blue/Black/Yellow
True/False
Military Veteran/Not A Military Veteran
Gender
Race & Ethnicity
Nominal data measure(s) of central tendency: Mode
Ordinal
Ordinal data is categorical data that has a logical order to it. It’s still categorical, so there is no inherent underlying numerical value. You can’t measure the exact “distance” between categories even though you can put them in order. We often assign a numeric value to them to make analysis easier.
Some Examples of Ordinal Data
Opinion: (1) Strongly Disagree, (2) Disagree, (3) Agree, (4) Strongly Agree
Educational Attainment: (1) Some High School, (2) High School Graduate, (3) Some College, (4) College graduate
Income Level: (1) $0-10,000, (2) $10,001-$20,000, (3) $20,001-$30,000, (4) $30,001 or greater
Ordinal data measure(s) of central tendency: median and mode
We often see the mean used with Likert Scale data. That's usually fine if you’re doing simple comparisons (e.g. customer satisfaction last year vs this year) and not doing rigorous data analysis (e.g. multivariate regression).
Interval/Ratio
Interval/Ratio* data is continuous data with an underlying numerical value. Interval/Ratio data can be rank ordered, and you can do mathematical operations with it because differences between values are meaningful and measurable.
Examples of Interval/Ratio data:
Height
Age
Annual income
Rent as a percentage of income
Years of Education completed
Interval/Ratio measures of central tendency: Mean and Median
The mode often not useful with interval/ratio data because there can be too many possible values. For example, if we asked 1,000 people for their income last year, we'd get several hundred different responses. In this situation, the mode would not be representative of the entire data set.
NOTE: Interval and ratio data are technically different levels of measurement. Their difference is not meaningful to most of us, most of the time, so we’ll combine them for simplicity.
Pro Tip: If you want to turn around a dull party, then try correcting someone when they mistakenly label interval data as ratio data in a casual data nerd conversation. Take Me Home, Country Roads suddenly stops. Harlem Shake plays. Wild dancing ensues. Works every time.
Descriptive Statistics
A descriptive statistics table presents an overview of your data. For example, you might display demographic characteristics of people using your pantry service along with data about meals served or the average number visits per year.
A descriptive statistics table helps you understand who you are serving and what/how much you are doing.
In Table 1(below), we present some descriptive statistics for our food pantry. Table 1 includes data on gender and race of the head, household composition, and finally a little performance data including: count of households served, total beneficiaries, and meals provided.
Table 1: Descriptive Statistics for Food Pantry Participants July 1, 2022 - June 30, 2023
Head of Household Gender | Count | Percent |
Woman | 83 | 67% |
Man | 38 | 31% |
Another Gender | 4 | 3% |
Head of Household Race/Ethnicity | Count | Percent |
Asian, Pacific Islander | 6 | 5% |
Black | 25 | 20% |
Hispanic or Latino | 25 | 20 |
Multi-Racial | 19 | 15% |
White | 44 | 35% |
Another Race or Ethnicity | 6 | 5% |
Household Composition | Count | Percent |
Single Parent/Guardian | 69 | 55% |
Multi-Parent/Guardian | 19 | 15% |
No Children or Dependents | 37 | 30% |
Households and Services | Count | |
Unduplicated Households | 125 | |
Unduplicated Beneficiaries (all household members) | 350 | |
Total Meals Served | 25,200 | |
Table 1 Description: example of descriptive statistics table for a food pantry. Table 1 includes participant demographics (gender, race, household composition), as well as some performance metrics (unduplicated households served, unduplicated beneficiaries, and total meals served).
The descriptive statistics table helps us quickly grasp the size, scope, and makeup of our program. For example, we can see that 67% of the heads-of-household that come to the organization are women. We can also see that slightly more than half (55%) of the households served are single-parent households. Finally, Table 1 shows that we served 25,200 meals to 125 unduplicated (i.e. unique) households.
Table 1 might be too large to grasp easily. Another option would be to break it apart. It would make sense to separate the demographic fields in the table from the fields focused on services delivery - basically everything under "Households and Services".
Questions to ask about your descriptive statistics table(s):
Are the people we serve what I expected...
based on our mission, neighborhood or service area, program goals/targets?
How does this compare to the same period last year?
Are we serving more men, multi-parent households, people who are employed full-time, veterans, etc?
What do I see that is unexpected?
Did we serve more or fewer meals last year? Is that change surprising?
Is there anything I want to dig into more deeply? (see next section on cross tabulations)
Cross tabulations
Cross Tabulations (aka crosstabs) are one of the simplest and most powerful ways you can analyze your data. They are particularly useful when you want to compare groups.
For example, you might want to compare the average length of enrollment in a program for single parents to participants who don’t have children. Or, you might want to compare program dropout rates between people who speak English as their first language vs those whose first language is not English.
NOTE: You can use pivot tables in Excel and Google sheets to produce cross tabs. There are countless videos online that can teach you to make them in under 10 minutes.
Looking at Table 2 (below) we compare the results of single parent and childless adults in our employment program.
Table 2: Cross Tabulation of Household Composition and Employment Program Outcomes for Program Participants Between January 1, 2022 and December 31, 2022.
Measure | Single Parent Count | Single Parent Percent | Childless Adult Count | Childless Adult Percent | Total Count | Total Percent |
|---|---|---|---|---|---|---|
Obtained Job | 80 | 64% | 50 | 67% | 130 | 65% |
Job has health benefits | 57 | 46% | 45 | 60% | 102 | 51% |
Job pays at least $15/hr | 68 | 54% | 47 | 63% | 115 | 58% |
Job has PTO | 56 | 36% | 44 | 59% | 100 | 50% |
| | | | | | |
Total | 125 | | 75 | | 200 | |
Table 2 Description: Example of a cross tabulation that compares single parents, childless adults, and all participants along 4 job program outcomes (obtained job, job has health benefits, job pays at least $15/hr, and Job has PTO). Table includes counts and percentages for each outcome measure.
We include both counts and percentages in our table because that gives us the clearest view of whether our work is equally successful for single parents and childless adults.
Table 2 also presents data for all participants. In this example, we only have 2 groups (single parents and childless adults). However, if you had 3 groups or more, comparing every group to every other group gets confusing and tedious. It’s helpful to be able to say “Participants who are X (e.g. single parents, immigrants, college graduates, etc) are/are not as successful as participants overall.
TIP: When presenting crosstabs, I like to put the differentiating variable in the columns. For example, if I wanted to compare program results based on gender, I would make gender categories my column headers. I would use the rows below to compare the results. In Table 2, that differentiating variable household composition (Single Parent vs Childless Adult vs All Participants).
Among X, what is Y? The power of cross tabs
Cross tabs let us ask and answer the general question “Among X, what is Y?”
Among the single parent households, what is the average number of times do they visit our food pantry?
Among multi-racial households, what is the average number of meals they receive each time they visit?
With that information, we can make simple comparisons that help us understand our programs and the needs of our participants.
How does the average number of visits to our food pantry compare for single parents, multi-parent, and childless adult households?
How long do employed participants receive services compared to unemployed participants?
We obviously can't answer every important question using cross tabs, but they can help us understand our programs' equity, effectiveness, and areas for improvement.
Scales
You probably won't see scales in your stats 101 course, but they're powerful and easy to understand. Scales allow you to combine a set of related measures into a single continuous measure.
Let’s say you have four Likert Scale questions about a person’s financial stability.
I can pay my rent on time
I save money for the future
I track my income and expenses
I can afford nutritious food
Our questions have the following response options:
Never
Rarely
Sometimes
Often
Always
NOTE: It's worth pointing out that the numeric values we assigned to each response are logically ordered just like the categories. But, the categories themselves are not precise and the numbers we assign are not perfect.
There's no reason that "never" should have the value of 1 instead of 0 or -1 or something else. The values on our scale should be evenly spaced (e.g. 1, 2, 3, 4, 5 not 1, 3, 17, 97, 98, 250), but we're still dealing with categories that are "fuzzy". It's fine, just be aware of it.
Scales give us three big benefits.
First, you can treat data in scales as interval/ratio data, so you can do calculate means (and standard deviations if you are so inclined).
Second, many concepts that we care about, such as financial stability, are complex and multi-faceted. No single question or data point can measure the concept fully, so we might need to use multiple questions to do that. With scales, we can combine data from multiple questions into a single “scale score”.
Third, scales can make cross tabulations much easier and more accurate.
In this example, we could create cross tabs with race and each of the 4 financial stability questions. Frankly, we probably should do that to see if anything stands out, but we also in tricky territory. We now have 4 tables to compare and synthesize to determine if our program is less effective for, say, Native Americans. That's a tough exercise.
But, we could combine the assigned numeric values for each of these responses into a single scale score. With a single scale score, our comparisons are much easier.
Table 3 and Table 4 (below) helps us see how we might construct a scale score.
Table 3: Individual Participant Responses to Financial Stability Questions
| Pay Rent on Time | Save Money | Track Income & Expenses | Afford Nutritious Food |
|---|---|---|---|---|
Jim | (4) Often | (2) Rarely | (1) Never | (2) Rarely |
Tina | (3) Sometimes | (1) Never | (1) Never | (1) Never |
KT | (4) Often | (2) Rarely | (2) Rarely | (3) Sometimes |
Maria | (5) Always | (1) Never | (3) Sometimes | (2) Rarely |
Pat | (5) Always | (4) Often | (4) Often | (5) Always |
Table 3 Description: Example of table that presents individual values to four Likert Scale questions focused on financial stability. The complete questions are listed above. For simplicity, the columns are labeled as: (1) Pay Rent on Time, (2) Save Money, (3) Track Income & Expenses, and (4) Afford Nutritious Food.
Table 3 is OK. We can get a sense that Pat is doing pretty well overall and that Tina is struggling. But, Table 3 is deceptive because it's small. There are only 5 participants and 4 questions. What if there were 50 participants and 10 questions - so 500 cells to review? It would be overwhelming.
We can make these comparison simpler by combining the values we see in each column into a single Financial Stability Scale score. For example, for Jim, we would add 4+2+1+2 = 9. Jim has a score of 9 on the Financial Stability Scale.
Table 4 shows the results for all 5 participants in the Scale Score column.
Table 4: Financial Stability Scale Score and Average Item Score
| Scale Score | Average Item Score | Comparison to Overall Averages |
|---|---|---|---|
Jim | 9 | 2.25 | Below Average |
Tina | 6 | 1.5 | Below Average |
KT | 11 | 2.75 | Average |
Maria | 11 | 2.75 | Average |
Pat | 18 | 4.5 | Above Average |
Overall Averages | 11 | 2.75 | |
Table 4 Description: Example of table that includes the Scale Score, Average Item Score, and Comparison to the Overall Averages for five participants on a Financial Stability Scale. Two participants (Jim and Tina) have below average scores, two participants (KT and Maria) have average scores, and one participant (Pat) has an above average score.
Looking at Jim's score of 9, do we think that's good, bad, or somewhere in the middle? It’s hard to say without more context.
Scales can strip away the context that is inherent to these questions (i.e. 1 = never, 5 = always), so we have to find a way to create that context again. We have two easy ways to do that.
First, we can just calculate the overall average financial stability scale score. That’s easy enough. We included that on the bottom of Table 4 as "Overall Averages".
Now we can compare each individual to the overall average score.
This is an improvement, but it still feels a little lacking. Maybe if I had access to a few years of data for comparisons, then it would mean something more. As is, it still feels like we’re missing something.
Second, we could calculate each individual’s average item score. To do that, we simply divide their scale score by the number of questions (n=4). Pat has a score of 18, so Pat's average item score is 8/4 = 4.5.
This gives us useful contextual information because can anchor our analysis on those original response values (1=never, 2 = rarely, and so on).
Tina’s average score is 1.5. Her average score lies somewhere between Never (=1) and Rarely (=2). Tina is probably not financially stable. On the other hand, Pat’s average of 4.5 is somewhere between Often (=4) and Always (=5). Pat's looking pretty good.
TIP for making Scales: The questions you include to measure a complex concept clearly belong together and are associated with that topic.
For example, if you are measuring financial stability, you would not include veteran status or dietary restrictions. Sure, you could find a way to say those things impact financial stability; but they are pretty far afield of the main concept you are measuring.
We can see how scales take potentially complex data and analysis and simplify it. They can help use draw conclusions about our work and focus on areas for celebration and improvement.
Data Tools and Data Software
Spreadsheets are powerful tools for data analysis, and there are countless videos and written tutorials that explain how to do cross tabulations (aka pivot tables) and basic statistical analysis. When your data gets more complex because your nonprofit has multiple programs, data stored in spreadsheets gets hard to manage; but they are affordable and powerful tools to rely upon as long as your data is simple.
If you’re using (or shopping for) specialized data collection or case management software it should have built-in tools that easily produce basic statistics that you regularly need, and/or you should be able to export your data to other data analysis tools (like spreadsheets).
Learn More About Nonprofit Data Management
This post is part of our nonprofit data bootcamp series. Check out the complete list of nonprofit data bootcamp topics with links to other published posts.
Reporting your impact is hard when you’re juggling spreadsheets. countbubble makes it easy so you can focus on your mission.
countbubble is case management simplified. We can help your nonprofit master data collection and reporting. Email us contact@countbubble.com or sign up for email updates on blog posts, product news, or scheduling a demo.
Founder, CountBubble, LLC
Σχόλια