Eventually someone will suggest that your organization must do a community development evaluation. The topic of assessing your organizational capacity and programs generates many self-righteous pronouncements on why you must do evaluation now, what that effort should look like, how rigorous it must be, and often why you must pay a consultant a princely sum for helping you do that.
With the exception of some funders that require community development evaluation at the conclusion of a grant project or reward rigorous evaluation processes with more funding, no one is forcing your organization to do anything that you do not find productive and useful in making decisions about how to spend your money and time. Do not let anyone shame you into entering into this process.
If you are not familiar with the terminology, we are not referring to the vital practice of personnel evaluation of staff members, but rather to the process of assessing how the organization itself is performing.
We should note that we are making an uncharacteristic number of negative comments on this page. We do wholeheartedly support the notion of an organization striving for continuous improvement. Any critical remarks here are aimed at what we see as an excessive amount of self-righteousness among those who think that program evaluation is just the ultimate in quality assurance for a community association.
But when your group itself wants to step back and discover how well it is performing its mission, it is good practice to delve into organizational evaluation a bit to see what would fit your needs. Sometimes you want to justify what you have been doing already, quantify your impact so you can explain it to the community better, shed light on whether you should expand or contract certain services or even the organization itself, support your case that your critics or negative publicity are wrong, or consider what a change of leadership or political climate means for you. Those are prime occasions for the topic of community development evaluation to arise.
We urge you to be very clear about why you are doing an evaluation, and to make sure that board members understand and support the rationale. If your real aim is to discover how to improve your work, make sure that everyone understands that this is not meant to be a judgmental process. If your goal is to generate facts and figures you can use to support your case with donors, be honest about that.
Having said all that, sometimes you have forced community development evaluation. In that case, just smile and make the best of it. Learning something new about ourselves can’t be a bad thing.
Before committing to a community development evaluation process, ask yourself whether you have the data needed, whether any change is possible on the basis of the evaluation, whether you will be able to do the evaluation yourself, and whether you will need to expend scarce resources to purchase consulting services. If you don't have the data you will need, if your funding agency won't let you change course even if you decide that you should, and if you can't afford a consultant, our advice is to put this project on the back burner until those conditions change.
We also oppose the tactic of hiding behind an organizational evaluation when what is really needed is conflict resolution or dealing directly with poor staff performance.
Also consider the fallout if the evaluation turns up one or many distressing facts. If your board and organization is the only consumer of the evaluation report, clear-eyed honesty is a great asset. But if your funders, governmental regulators, and organizational competitors are likely to obtain access to the evaluation, be careful about including elements that you can predict will show your organization in an extremely negative light.
Many organizations have had a bad experience with evaluation, perhaps because the process was too lengthy and time-consuming, it felt negative and uncomfortable, or it cost too much. If this is your history, it is important to recognize that up front and to discuss thoroughly what happened earlier and how to prevent a repetition. If board members with long involvement remember these earlier uncomfortable situations, they are likely to sabotage the new effort or merely be lackluster in their support of the community development evaluation process or the implications of its results.
Although community development evaluation can be maddeningly subject to fads, current practice could be broken down into:
This same framework can be applied to large and small social service organizations, by the way.
In internally focused evaluations, relevant questions are:
One versatile on-line example for this internally-focused evaluation of capacity is from the Greater Twin Cities United Way. You can add or subtract from their questionnaire based on your interests and needs.
The second type of community development evaluation focuses on outputs, or what is often called performance evaluation. The objective is to determine the reach or coverage of your programs, and how efficient and how productive your programs are. Confusingly, this type of evaluation sometimes is called process evaluation, meaning a review of whether program activities have been carried out. This is contrasted with an outcome evaluation, which will be discussed in the next section.
For what we are calling an output evaluation, output numbers must be collected for each and every program that you wish to include in the review. Relevant questions include:
The third type of community development evaluation points to the newer, more trendy emphasis on program outcomes. An outcome is a statement of the net result of the inputs and outputs. Ways to paraphrase this concept are:
To understand this subtle shift from output to outcome, let's look at the example of a meal program for homeless or low-income people. The output may be easy to count: for example, 144 breakfasts and 161 dinners are served on an average weekday. Asking about the outcome is more complicated in this case. Was the outcome a net increase in the ability of your population to be productive in a job search? Was the outcome better health because nutritional needs are being met? Was the outcome less petty theft at the convenience store? You see how the outcome questions quickly can become either profound or silly.
So in the end, another way to think of this is that considering outcomes helps your organization figure out if your outputs were enough. Output thinking is fine, but sometimes increasing that output number becomes our only objective, whereas really we should be thinking about whether the outputs are leading to the desired overall outcome for the people or place we're working with.
If you are affiliated with a broad-based community organization aiming to impact every aspect of your community or neighborhood, as we suspect most of you still reading are, you can look at community indicators as a measure of your outcomes. But if you are specialized in health, alternative transportation, social services, or real estate development, looking at very broad community indicators may give the wrong impression of how your organization is doing.
Whereas outputs almost always need to be measured for each program, sometimes an outcome can be addressed in one overall summary measure, such as reduction in homelessness.
Both output and outcome analyses rely heavily on numbers and statistics. If you feel weak in this area, you might benefit by reading about neighborhood demographics or our reply to a question about monitoring we received from Africa.
The biggest problem with formal evaluation is the time and money required. If the time commitment that most consultants prescribe is too heavy for an organization, this is a good time to consider what is now being called rapid appraisal. Rapid appraisal techniques include interviews with key stakeholders, focus groups, facilitated community meetings, field observation of programs or behaviors, or mini-surveys, which now may be administered online.
We think that smaller organizations with lesser means, such as neighborhood associations, might consider forming teams that trade evaluations with other neighborhood groups. Such exercises might not rise to the level of scientific rigor, but they could be eye-opening for both community organizations.
Another tip for reducing the cost of community development evaluation is to consider oral reporting or requiring a summary PowerPoint presentation, rather than the somewhat lengthy report that most consultants write. If you are undertaking the evaluation primarily for your own information, this can be a viable option. Also you can look for university professors who may be able to produce your evaluation economically, sometimes using student help.
One key to success in community development evaluation is to avoid the temptation to abandon the evaluation process and jump right into problem-solving the minute that an area needing improvement is discovered. Certainly use any intelligence you uncover right away, but don't stop evaluating mid-stream if you had a good thorough discussion of what you are trying to accomplish with the evaluation process.
Internationally many world organizations speak of monitoring and evaluation as one unit. The best introduction to "M&E" is from the World Bank. The principles we have been speaking of apply at the nation-state level as well, but obviously the scale will have to be adjusted and the effort definitely requires an NGO or consultant to conduct an objective study.
Mature neighborhood or community associations and community development corporations need a considerably more sophisticated discussion than what this short overview can provide. A good starting place is evaluation resources provided by NeighborWorks. CDCs and larger community organizations based in the U.S may benefit from Enterprise’s self-assessment tool. Finally, to figure out whether you are ready for evaluation, you may want to look through this framework for evaluation checklist, which is educational in itself.