The discussions of outcome measures are combined in sequential order instead of reverse chronological:
Outcome measures – are you just running around in circles?
Are you changing the world or are you just running around in circles? What change is happening because of all your effort? That’s what we mean by outcome measures. Many donors are now asking what are you accomplishing? instead of what are you doing?
What is the difference?
In an educational setting that is the difference between students just sitting in chairs compared to actually learning something. Donors want to hear what you changed, not just what you did.
There is a difference between financial inputs, program outputs, and outcomes. For example, in a pregnancy resource center, spending $500,000 on salaries and all the other stuff to run the program is a financial input. Meeting with 1000 clients, providing 500 counseling sessions, and administering 300 pregnancy tests is an output. Being in situation where 100 of the clients walking in the door are fully intending to have an abortion when the arrive but instead decide to carry their child to term is an outcome.
Donors don’t want to hear about the financial input and they are slowly realizing that the output may or may not have any real meaning. In the pregnancy center example, let’s say the 300 pregnancy tests you provided were given to clients who would have carried to term whether or not they ever visited your center. If that is the only outcome from your ministry, then donors are very justified in asking why they should support you.
The same type of questions can be asked of any non-profit organization – what are your outcomes?
Next discussion – measuring outcomes is difficult
Outcome measures – Measuring outcomes is difficult
It is easy to measure how much stuff you did. It is very hard to measure what you actually accomplished.
It is easy to count:
- how many counseling sessions you provided to clients,
- how many people sat through a worship service,
- how many at-risk children attended your summer camp, or
- how many nights of lodging and days of counseling you provided to people at your substance abuse shelter.
It is really, really hard to quantify:
- how many clients were abortion-minded when they arrived at your clinic but actually carried to full-term
- how much has the spiritual maturity of the congregation improved in the last year, or the last decade
- how much has the level of substance abuse gone down and how far has academic performance gone up for the at-risk children who attended your camp last summer, or
- how many addicts starting treatment at your shelter are now free of their addiction and living in stable circumstances.
Those are the changes that you yearn to make. Making those changes is why you try so hard and give so much of yourself. You already know how incredibly hard it is to make those changes. Yet you have the God-given ability to do so and you probably see lots of those kinds of changes.
The funny thing is you probably aren’t too interested in the raw number of pregnancy tests administered or the occupancy load in the shelter (that’s the percentage of beds filled each night). Do you really, really care about those counts? I doubt it.
What fires you up? What keeps you going when you’re discouraged? I know the answer and it’s the same as the number you need to chase for outcome measures. It’s the babies you see years later that would have not been born except for your center or the people who permanently kick the habit and keep a solid, honest, paying job for years. That’s why you do what you do. The question is how to quantify that.
It is really hard to put those kinds of changes into numbers. We don’t have a good handle on knowing how to find out those things and we don’t have usable information systems to accumulate the information even if we could count it. But that is the challenge all of us need to take on.
Next post: can you measure spiritual maturity in a local church? Previous post: are you running around in circles?
Outcome measures – Can you measure spiritual maturity in a local church?
Yes. It is possible to quantify spiritual maturity in a local church. Every non-profit along with all churches can learn from that answer.
All non-profit organizations should listen carefully to learn that it is possible to measure these types of intangible factors.
Are you putting time and effort into stuff that is helping people in the church grow and become more spiritually mature or are you just pushing spiritual busy-work? There is a group that has been able to use modern consumer-survey techniques to quantify the levels of spiritual maturity of individuals in a local church. A research team has labeled four levels of maturity, each of which has its own behaviors:
- Exploring Christ
- Growing in Christ
- Close to Christ
- Christ Centered
Through surveys, the team can categorize people into a specific category. Some overviews of their work are here and here.
What are some of the astounding insights? For me, here are just a few of the remarkable things the research team found:
- The level of church involvement has no correlation to spiritual maturity – in other words, doing lots of “stuff” at church doesn’t seem to have any impact on spiritual growth.
- Lots of people are stuck at level 2 – 22% of the people surveyed are stuck. If those of us in leadership could figure out what is behind people being stuck, and provide them a helping-hand so they can break out of those patterns, they could start growing again.
- Most of the evangelism, giving, serving, and praying is done by people who are at the highest level. If you want to change how much praying, giving, and serving is taking place in your body of believers, then grow your people.
- A lot of people at the most mature level are discouraged and becoming disengaged. That is an unsettling waste of time and talent – (combine this idea with the previous point to figure out the huge impact of this disengagement). If we in leadership can help the discouraged to re-engage, can you imagine the increased kingdom impact that would follow?
You can get two superb books to discuss this at the Willow Creek Association web site, Reveal: Where Are You and Follow Me: What’s Next for You.
Don’t know whether you like the Willow Creek community, or can’t stand them, or are indifferent. It doesn’t matter. One of the most amazing finds from their research is that the results are consistent across denominational boundaries. I have learned that a large congregation in my denomination was one of the first participants in the survey. Thus, my conclusion is that particular congregation follows the patterns described in the book. In addition, I would infer that my local congregation does as well. All of us can learn from these books.
Previous post; are you running around in circles and measuring outcomes is difficult. Next posts: why does this matter to churches? and what difference does this make to your NPO?
Outcome measures – Why does it matter to churches that it is possible to measure outcomes in a local church?
Helping people grow spiritually — Every church I’ve every worked with had a vision that included some variation of that idea. It is phrased differently with varying focus, but that seems to be typical.
What if you could identify things that help that growth? What if you could identify things that are completely irrelevant to helping people grow?
If you could do that, I bet you would put more effort on the things that help growth. I bet you would cut back on the stuff that either gets in the way or is just fluff. None of us have time for the fluff.
It will be extremely hard work, but the tools discussed in the previous post give us a path to sort out what works and what is a waste.
There is a saying about advertising – half of your advertising is wasted – you just don’t know which half. What would a business do if it could sort out which parts of the ad campaign was working and which was a waste?
What would you do if you could figure out what is hindering spiritual growth and what is firing people up? Wouldn’t it be worth the effort to find out?
Next post – why does this matter to NPOs? Previous post here. You can read the outcome measure posts in chronological order here.
Outcome measures – Why does it matter to other NPOs that churches might be able to measure spiritual maturity?
The most exciting thing to me from Reveal: Where Are You (discussed here) is that it IS possible to measure such things as spiritual maturity. You CAN quantify that outcome measure in a local church.
Opening that door in turn opens the door to measuring other intangibles.
It’s an incredible encouragement to think you could find some way to measure those types of outcomes in your NPO. I have heard of creative ways to statistically measure the impact of abstinence training. One of my clients has several research projects underway to quantify the impact of their training programs in Africa and Asia.
If you work with addicts, isn’t the deepest desire of your heart to get people away from that junk and give them tools to stabilize their lives? How wonderful it would be to figure out how to track individuals and quantify the long-term results of your efforts. You can apply this measurement concept to whatever program focus you are working with.
The even more incredible opportunity, after we can measure outcomes, is ask ourselves the even harder question of what worked and what didn’t work. What parts of your program are getting in the way of making a change? If we can find out, then we can dump the stuff that is wasted effort and put all our focus on how to change the world. How incredible would that be?
Other posts in this series here, here, and here. You can read the series in chronological order on the Outcomes page in the header of this page.
These posts are copyright © 2010 James L. Ulvog