Outcome Management

Outcome Management

Outcome Management is a performance measuring tool.  It says that the achievement of the target is the driving force.  On the way to reaching it, there are usually several stages - called milestones - where progress checks can be taken and adjustments made or new ideas introduced.  It's just like map reading on a journey - you might go slightly off-course but can rectify problems on the way and still get to your destination.

In order to measure improved outcomes, you have to start by quantifying goals, setting milestones and put them into a context to create a plan for performance improvement or organisational change.

Outcomes - Getting to Results

The most important reason for a service to focus on outcomes is not to evaluate their service, or even to hold them accountable. It is to increase the probability that outcomes will happen. Evidence abounds that projects with clear aims and objectives and a focus on outcomes will out perform over projects that lack these qualities.

 

Outcomes are specific changes in behaviour, condition and satisfaction for the people that are served by a project or a service. These gains are generally signal improvements or ‘human gains' that have been brought about by the service intervention.

 

In some health and social service areas gains and outcomes are not so readily discernible. In some cases outcomes or ‘success' is difficult to define. We often know what we are trying to change but the method of measurement is difficult. In some cases we have difficulties verifying client changes, especially if we only have sporadic contact with them. We also get perplexed about issues of ‘attribution', and whether it was our organisation that made the difference or some external influence. Not to mention how long the outcomes or gains remain in place after our work is done!

 

Perhaps in part to side step these issues, governments, funding authorities and service providers often move from a result focus to an accountability focus. This moves the looking glass to what can most readily be counted. As a result we see an emphasis on procedural compliance and outputs and activities. Did we see the right number of clients? Did we fill up our beds as planned? Did we deliver the correct number of counselling sessions?

 

The problem from both provider and investor perspectives is that compliance with procedures and completion of inputs does not guarantee the outcome for the client served. If a group holds workshops for inner-city kids who are unemployed, the outcome of interest is not workshop completion or even whether participants found the workshop helpful or enjoyable. It is the number of participants who get and keep a job!

Beyond accountability lies the solution called "evaluation". Those concerned with understanding outcomes may turn to evaluation specialists, who prescribe comprehensive and expensive assessment methodologies. Such methodologies are assuredly needed for precise measurement and to meet the canons of quantitative social science research. But, the cost of evaluation can exceed the cost of the project!

The results focus is an alternative to procedural accountability and formal evaluation. It asks those who wish to undertake a project to take responsibility for specifying the consequence of their intended activity, then to play a lead role in assessing the extent to which those consequences happen.

 

The definition of results is a three-stage process:

  • 1. Define the impact areas in which you expect or hope to see change.

For example a group may well rest their initial thinking on the activity level. "We will deliver ten counselling sessions to 25 women who live in abusive environments". Our first challenge is to help define the impact areas for which counselling is a means. While counselling is certainly a fine thing to support, we are interested in what counselling will yield. The group might indicate that the outcome is the ability either to stay in the situation and turn it around or leave an abusive partner. Fine, we have a starting point.

Often, the impact question requires that you get beneath the broad words such as "quality". Fund us to do "quality day care" or "quality respite" for families with a person with disabilities. Fine, we're always pleased to fund quality. But in this case what does it mean? What is the outcome of quality day care? You may learn that it means helping people to play, relate co-operatively to other children...or increase the chances that the parents will be able to get jobs. Whatever the outcome, you have a starting point.

  • 2. For each impact area, what indicator will confirm or at least signal the presence of an impact.

Often this question can be answered by looking at the sequence of progress a client is anticipated to make through the project intervention. In the counselling example, the agency might define the following general progression:

  • the women counselled will begin to articulate choices and options in their lives - at first in little things, then bigger ones;
  • the women counselled reach a point where they can make an informed choice between clear options of staying in a situation and turning it around, or leaving the setting in which they are abused;
  • regardless of option taken, the abuse stops and the women report a higher quality of life as a consequence.

At this point, you are ready to get to indicators. How will we know that the women counselled begin to consider options? One way might be that the person could indicate at least three incidents in the past week in which they could identify a decision point where they felt they had at least two options. For the major decision point of "stay or leave," some type of client delineation of the pros and cons of each choice might be a starting point. At the final level, the indicator is clearer: the abuse stops or it does not. While it may prove difficult to measure this indicator (as in the case of psychological abuse), at least the desired outcome is clear.

In general, specify indicators that are as strongly related to the real outcome as possible. At times, a surrogate for final result will be needed. For example, if one assumes that a reason for teenage pregnancy is unemployed young men, then increased jobs and job retention might well be a result surrogate.

Getting to behavioural terms is also important. If we say that the indicator of a positive impact in human relations is "better communications," we have no way of measuring improvement. We must specify what kinds of behaviour will signify better communications. Similarly, if an indicator is higher self-esteem, we should specify the behaviours that will indicate its presence.

 

  • 3. Specify how indicators can and will be used to determine the performance level achieved.

While it would be nice to have resources available to focus on measuring results, this is simply not possible for most projects. The following guidelines are suggested:

  • The responsibility for showing evidence of reaching outcomes should lie with the implementing group, not the investor. Indeed, the primary point of engaging in verification of results (which is different from explanation or even documentation) is that this will help to bring results about.
  • The implementing group should focus solely on those individuals they will specifically include, not on a broader category of people with a given need or problem. As a direct service provider, you are not trying to solve teenage pregnancy in South London with £20,000. You are trying to gain a given reduced level of pregnancy among the 45 persons to be helped in your programme.
  • Develop where needed and possible baseline data about your programme clients and their behaviour prior to the start of the programme. An assessment interview is always a good starting point to generate better data.
  • Support strategies in which clients help to report information on the impact indicators chosen. They can keep logs of behaviours and other matters, for example. And since it is their perspective that often matters most in the outcome, their view as to what is happening is often the most important one.
  • In many areas of social and human service, the definition of outcomes may come more readily than with counselling. We chose this example precisely because it is so difficult. Other factors that make the definition and verification of outcomes complex and even illusive include differences in counselling style and approach, the privacy to be accorded an abused woman, and sensitivity in entering the home environment. But this reality is not a reason to abandon the pursuit. Indeed, in the area of many social problems, the need to go beyond present performance based on lofty goals and "best efforts" is particularly important.

Results can be left at the level of short-term intended consequences. Other dimensions, however, may need consideration. Here are some potentially important ones.

Result longevity

Some results may be strong and fade quickly once the treatment or intervention ends. In other cases, results will survive the conclusion of the programme that helped yield them.

Unintended outcomes

Some results that prove most consequential may not have been intended or anticipated at all. In some areas, their impact must be noted.

Adverse consequence

We presume too readily that good results always follow good intentions. Not so. Virtually every initiative has some capacity to do harm, albeit inadvertently.

Benefit distribution

Total result level (aggregated from all clients or participants in such terms as average gain) may be less important than the distribution of improvement. In virtually all programmes, some people gain far more than others. This is important to consider.

For outcomes to become a focal point for driving a project, they must be specified in advance and monitored on a regular basis. Providers and investors need to work together to become comfortable with a focus on outcomes and to approach the subject with a sense of learning and enquiry about ‘what works'.

Outcome frameworks that that start from the premise of learning and continuous improvement will always out-perform those that focus on accountability and punishment.