Your nonprofit’s mission statement demands that you do something, rather than nothing, to change the world for the better. It gives you a reason to exist, and often a mission statement doesn’t tell you exactly what to do.
That something is meant to “empower”, “provide opportunities”, “create equality”, “increase achievement”, and more. These terms inspire us to achieve great things, but their meanings are not very precise. The goals that nonprofits work toward can also be vague. Outcomes like “housing stability”, “self-sufficiency”, and “academic-achievement” leave a lot of room for interpretation.
The loosely defined terms we use in nonprofit mission statements and goals are called concepts. Concepts have meanings that we generally agree on, but those meanings have a certain fuzziness to them. Your definition of "achievement" or "discrimination" is probably different from mine. In fact, I might even struggle to provide a decent definition of those terms if asked for one.
How do we "measure our mission" with all these fuzzy concepts?
We must first precisely define the concepts we use in our mission statements and goals (conceptualization), and then we need to find good measures for those precise definitions (operationalization) to track our work effectively.
In this blog post, we’ll talk about conceptualization and operationalization so that you can learn to convert vague concepts into thoughtful, useful measures of your nonprofit's programs and services.
Nonprofit Missions Can’t be Directly Measured
Let’s look at a simple mission statement:
To support thriving families and high academic achievement for low income youth.
Almost every word in this mission statement is a concept that should be clearly defined. We can pull out the concepts and see what’s left:
To... and... for...
Everything else is a concept.
Don’t believe me? Define each of these concepts, ask a colleague or board member to do the same, and then compare your answers.
Support
Thriving
Families
(High) Academic Achievement
Low income
Youth
This is the starting place of our measurement challenge. There's no perfect way to measure “Thriving Families” or “Academic Achievement” because those concepts can mean so many things.
If every important word is open to interpretation, how are we supposed to measure our success?
This is where conceptualization and operationalization come in. Your nonprofit’s mission inspires the programs and services you deliver, and likely the first thing you want to define clearly. Nonprofits need to clearly define goals and desired outcomes as well.
Conceptualization and Operationalization
Step 1
Conceptualization - Define a concept clearly and precisely.
Step 2
Operationalization - Identify the way(s) to measure a clearly defined concept.
Conceptualization
Let’s say that our nonprofit helps families achieve self-sufficiency. Most of us have a sense of what “self-sufficiency” means, but we need to use a clear and precise definition of this concept to make it measurable.
Self-Sufficiency is:
Able to get by on your own...
No, that’s not any better than “self-sufficiency”. It doesn’t provide any more clarity. We can do way better.
Able to pay your bills and meet your needs...
That’s a start. “Paying your bills” is a fairly concrete activity. But we aren’t that close yet. What if you pay your bills with $500 you inherited? You’re not likely to get lots of inheritances to keep you going. Also, does “pay your bills” mean the act of paying bills, or is it focused on having the money to pay your bills? Let’s keep trying.
Able to pay your bills with a combination of wage income and savings without the need for financial support from others...
Now we’re getting somewhere. We say where the money comes from - our income and/or savings - and that we are able to pay our bills without outside financial support. So, no SNAP benefits or financial support from family members. Based on the wording, we can assume that “pay your bills” is focused on having enough money to pay bills, not the physical act of paying bills (e.g. writing & mailing checks, logging into online accounts, setting up auto-pay, etc). We could certainly be clearer about that if we're worried about the ambiguity.
This might not be a final version of our definition of “self-sufficiency” but it’s starting to shape up into something useful.
What do you mean by useful?
A definition of a concept is useful when:
When multiple people read it, they have very similar ideas of what it means. There are few/no remaining questions about terms. In other words, it’s precise, clear, and has addressed as much ambiguity as possible.
You can create measures for it.
Dimensions of Concepts
Our definition of "self-sufficiency" has uncovered an important challenge of conceptualization. The concepts that we define, including self-sufficiency, can have multiple dimensions. In other words, concepts often have many aspects that can be used to define them.
We could define self-sufficiency only in terms of finances and that would be OK. We could also say that self-sufficiency includes physical and mental health, access to healthcare, housing, transportation, and lots more.
A person who is couch-surfing with friends and family might be able to meet all of their expenses with their income, but would our nonprofit define them as “self-sufficient”? Probably not. The challenge, then, is deciding which dimensions we care about. We include those dimensions in our definition of the concept.
Returning to our example, our nonprofit thinks that "self-sufficiency" is multi-dimensional. It encompasses more dimensions than paying bills on time this month. For us, self-sufficiency includes (1) finances, (2) housing, and (2) demonstrating consistency over time (i.e. stability or longevity).
We add these dimensions to our definition of self-sufficiency.
A person is self-sufficient when they:
1. Are able to pay their bills with a combination of wage income and savings without the need for financial support from others.
2. Have affordable housing and are not considered homeless.
3. Have maintained conditions (1) and (2) for at least the past 6 months.
Be Clear and Precise
Our simple definition of self-sufficiency includes the condition: “without the need for support from others”. However we could also define someone as “self-sufficient” when they receive public benefits like SNAP or regular financial support from a family member. That definition could be appropriate given our nonprofit's mission, goals, and participants.
This is a judgment call that our nonprofit needs to make. We can't ignore it because this detail matters.
In this case, we clearly state that “support from others” includes public benefits and regular monetary gifts from friends and family. We will also specify that Social Security, SSI, and SSI are acceptable forms of income for our definition of self-sufficiency.
So we can revise our definition once again.
Our definition of self-sufficiency is getting closer.
A person is self-sufficient when they:
1. Are able to pay their bills with a combination of wage income and savings without the need for public benefits (with the exceptions of Social Security, SSI, or SSD) or regular gifts from friends or family.
2. Have affordable housing and are not considered homeless.
3. They have maintained conditions (1) and (2) for at least the past 6 months.
Of course, there's more work to do. We should define "affordable housing" and "homeless". Those concepts leave lots of room for interpretation and need further clarification. You get the idea.
Tips for Conceptualization
Conceptualization can be tricky. Here are some tips on how to create your definition.
Look around at what others have done. There are likely plenty of nonprofits the type of work that you do. Learn how they define concepts that are relevant to your work and use their examples as a starting place. For example, the Arizona Self-Sufficiency Matrix is a widely used, multidimensional scale used to measure self-sufficiency.
Ask “what do you mean by that?” as you work through a concept. What do you mean by “pay their bills”? Are you talking about having enough money or the act of paying the bills? What do you mean by “without the need for support from others”? Drill into every word or phrase until you’ve created a definition that is clear and precise. (see Jaccard and Jacoby, Chapter 5)
When you have a definition that looks good, show it to close supporters, other staff, and people who aren’t connected to your work. Ask them to explain it back to you. You have two goals here:
First, you are trying to determine if your definition is clear and precise.
Second, for certain people (e.g. board members or your supervisor), you want to know if they support your definition.
Your definition does not have to be ground-breaking or unique. The way you define concepts could make your nonprofit stand out from the crowd, but your services and achievements are probably more important. Your top priority is to create definitions that are useful and appropriate for your nonprofit.
Operationalization
So, we have a decent definition of “self-sufficiency”, and we need to find a way to measure it.
When we operationalize our definition, we find ways to measure it. We identify measures (sometimes called metrics or variables) that we can track in paper files, spreadsheets, or service tracking data system.
Dimensions and Measures
We should find measures for every dimension of our concept that we've identified. Our definition of "self-sufficiency" includes dimensions about finances, housing, and consistency over time, so we need measures to capture each of those dimensions.
Single vs Multiple Measures
Sometimes we can use a single measure to measure a dimension of a concept or an entire concept, and sometimes we need multiple measures to track each dimension of our concept.
If we think that a dimension is not well captured by a single measure, then include multiple measures of that concept.
For example, if we wanted to measure academic achievement of students in our mentoring program, we might include measures about grades, standardized test scores, and teacher feedback. We could combine those three measures into a single academic achievement scale, with each representing about one-third of the total scale score.
Alternatively, we could decide that standardized test scores are the focus of our programs, and that single measure of academic achievement is sufficient.
Level of Measurement
We will need to select a level of measurement that makes the most sense for what we are tracking. Many things can be tracked in multiple ways, and there may not be a single "best approach". For example, we could track educational attainment by asking:
How many years of education did you complete? (Interval/Ratio)
What is your highest level of educational attainment? (Ordinal - Less than HS, HS Grad, Some college, etc)
Did you graduate high school (Nominal - Yes/No)
Think about the following issues when selecting the level of measurement of measures:
What is relevant to your programs and participants?
Does a person's number of years of education impact their program eligibility, or does their status as a high school graduate matter more?
Is your target population people with less than a college degree, less than a high school education, or something else?
Can you analyze the data effectively to answer important questions about your programs or your participants?
Is years of education meaningful if your participants come from multiple countries?
Can you collect the needed data consistently and accurately?
Do you think you can accurately track years of education?
Do you think that your participants can accurately recall it?
Is it worth your time to get this level of detail?
Remember, we can always convert data we've collected to a level of measurement with more detail (e.g. from interval data to ordinal data) to less detail, but not the other way around.
Operationalization Example - Self-Sufficiency
Let’s operationalize our definition of self-sufficiency.
A person is self-sufficient when they:
1. Are able to pay their bills with a combination of wage income and savings without the need for public benefits (with the exceptions of Social Security, SSI, or SSD) or regular gifts from friends or family.
2. Have affordable housing and are not considered homeless.
3. They have maintained conditions (1) and (2) for at least the past 6 months.
We operationalize our definition of self-sufficiency by finding measures for each of our dimensions.
1. Are able to pay their bills with a combination of wage income and savings without the need for public benefits (with the exceptions of Social Security, SSI, or SSD) or regular gifts from friends or family.
Option 1: Ask a question to measure this dimension
During each of the past the past six months, did you have enough money from income and savings to pay your bills?
Did you pay your bills each month?
Option 2: Gather relevant data to measure this dimension.
Monthly Income in each of the past 6 months excluding SNAP, Section 8, and other public benefits.
Monthly Expenses in each of the past 6 months.
I prefer Option 2 because it does not rely on the participant's recall of the past 6 months - which is difficult for all of us to do accurately - and it provides a lot more insight into their finances.
2. Have affordable housing and are not considered homeless.
Note: We should have defined "affordable" and "homeless" in the conceptualization process, but we didn't. As a result, these concepts are difficult to measure. For the sake of this example, we will define these as:
Affordable: Spends 30% or less on rent or mortgage.
Homeless: Lives in an apartment/house & has a lease/mortgage with their name on it.
Option 1: Ask questions to the participant
Do you think that your housing is affordable?
Do you live in an apartment/house & have a lease/mortgage with their name on it?
Option 2: Gather data and ask questions to the participant
Does the participant spend 30% or less of their total monthly income on rent or mortgage?
Do you live in an apartment/house & have a lease/mortgage with their name on it?
Please provide a copy of your lease.
I prefer option 2. While a participant's perception of affordability is meaningful and valid, it's helpful to follow a more objective standard (e.g. 30% of income). And, if we collected the financial data in item 1, we will be able to figure this out with some simple math.
3. They have maintained conditions (1) and (2) for at least the past 6 months.
Option 1: Ask direct questions to the participant
For the past 6 months, have you been able to pay all of your bills on time?
For the past 6 months, have you lived in affordable housing?
Option 2: Use previously data to determine the answer
If we gathered 6 months of data from above, we can determine whether the participant has met conditions 1 and 2 consistently for past 6 months.
I prefer Option 2 again. If we've gather detailed income and expense data, then we can determine whether the person has been met conditions 1 and 2 for the past 6 months. Again, Option 1 relies on the participant's recall of the past 6 months, and that's difficult to do accurately.
Self-Sufficient, Not Self-Sufficient, or a Little Bit Self-Sufficient?
We've defined our concept (self-sufficiency), created measures of multiple dimensions, and now we need to finally make a determination about the participant's self-sufficiency. We're faced with another decision.
The measures in our example imply that self-sufficiency is a "Yes/No" concept. "You are either self-sufficient, or you are not self-sufficient". However, it's probably reasonable to use a continuous “self-sufficiency score” that could range from zero to thirty, or a categorical rating such as low, medium, high (or probably something less judgmental). We could award points for each dimension of our concept and those would add up to a total score, or we could identify ranges for that score (low = 0-12, medium =13-25, high = 26-30) to make categories.
The point is that we have flexibility in determining our measures. Our measure of self-sufficiency (yes/no vs score vs categories) should be relevant to our work, something we can analyze effectively, and based on data we can collect consistently and accurately.
Tips for Operationalization
Once again, look around at what other nonprofits are doing. Look at your peer organizations locally or around the country to see if they have anything on their websites or annual reports that could be helpful. You can also reach out to your networks to see if they have any tips or standard measures they are using.
Search to see if there are any standard measurements of your concept. For example, you might be measuring food insecurity and a quick search of “food security measurement” will give you the United States Department of Agriculture’s standardized questionnaire that measures food insecurity.
Measure each dimension in your definition. If you think something is important enough to describe in your definition, then you probably need to measure it. For example, if you are focused on increasing girls’ confidence and success in STEM, then you might say a key part of what you are interested in is self-efficacy in math, which increases the likelihood that girls go into STEM. Then, you should probably create a measure of your participants’ math self-efficacy, their interest in STEM, and even plans to take STEM courses. It can take awhile to disentangle all of the dimensions in your definition.
Phone a friend. Your ultimate goal is to operationalize something that 1) makes sense and 2) you can measure. Once you have a draft, it is a good idea to run it by several people to see if it does make sense and if the way you want to measure it will work. I like to run things by colleagues and experts but also my mom. It should make sense to an expert and a non-expert to know you have clearly refined your idea and have a direct way to measure it.
Is All of This Really Necessary?
Maybe not, but probably so.
If your mission clearly states that you will provide food in a food pantry, no questions asked, then you might not have to go through this process.
But, there's a very good chance that you need to do some of this work.
For example, what will you count when you give away food? Will you count “bags”, “pounds”, “meals”, “servings”, something else? If you are counting bags or meals, how would you define a “bag of food”? It’s probably not “whatever goes in a bag is a bag”.
Also, how will you determine how much food someone gets? Does everyone get the same amount, or would you adjust the amount for family size?
How do you define a family? Based on that definition, how will you measure a family?
Even with such an apparently simple and clear mission, we still (probably) have to make the effort to define concepts and create meaningful, relevant measures.
Conceptualization and operationalization will take time and effort to do well; but they are worthwhile if you are having trouble communicating your work clearly or measuring your success.
Measurement
Now that you have clearly defined measures, you need to implement a system to track data. You can use paper files, spreadsheets, a relational database, or specialized data tracking software.
The tools and processes nonprofits can use to track and analyze data will be covered in another post, but you've taken major steps to being able to track your work (i.e. measure your mission) effectively.
Helpful Resources
I used two helpful resources while writing this post:
The Practice of Social Research, by Earl Babbie, is a good starting place. This is beginner friendly and is widely used in undergraduate social science research methods courses.
Theory Construction and Model-Building Skills, by James Jaccard and Jacob Jacoby. The book is a little more advanced but is full of practical advice.
You can probably find used versions of these books online for under $20. No need to get the latest edition. They aren't focused on nonprofit data specifically, but they can still be helpful.
Learn More About Nonprofit Data Management
This post is part of our nonprofit data bootcamp series. Check out the complete list of nonprofit data bootcamp topics with links to other published posts.
Reporting your impact is hard when you’re juggling spreadsheets. countbubble makes it easy so you can focus on your mission.
Learn how countbubble helps nonprofits measure their impact. Email us at contact@countbubble.com or sign up for email updates on blog posts, product news, or scheduling a demo.
Founder, CountBubble, LLC
Comentarios