Evaluation - Why Do We Do It? (#AllianceConf Part 1)

You are here

blandit

Let me start by saying it is really REALLY hot in Palm Springs. I'm not sure my brain can actually function in this type of heat. I am now more convinced than ever that Vancouver is, actually, heaven on earth. At least you can walk outside during the day, anyway. But other than that, I am enjoying the annual Alliance Conference very much, though it's just getting going.

Yesterday I attended the pre-day for those who work at MSO's (Management Support Organizations). Basically, that's organizations whose missions are to build capacity in the sector through training, education and consulting. Conversation focused a lot on the current economic crisis and how MSOs are and should be responding for themselves and for their clients. Many interesting themes came out of the overall conversation - one of which was evaluation.

Evaluation is something I've been struggling with lately, as I work to assess and refine how we do evaluation at Volunteer Vancouver. It's certainly not easy - and everyone is weary of the many many surveys coming out these days, so the "easy" answer doesn't seem quite so easy any more. Meeting attendee Peter York (TCC Group) made an interesting point - if MSOs don't focus collectively to determine what kind of metrics we should collect on our own activities - someone from on high will dictate that information to us. So true and so scary.

So it was interesting to me when evaluation came up again at the opening plenary of the conference later that evening. The speaker was Fran Barrett of Community Resource Exchange, winner of this year's Founders Lifetime Achievement Award (congrats, Fran!). She was talking about Head Start - the much lauded and highly respected school program from the US. Head Start is a model organization when it comes to evaluation - they can prove that what they do works - in those expensive longitudinal ways that most of us only dream of. They have proven, scientifically and without a doubt, that dollars spent on Head Start change the lives of their young clients for the better.

And yet, Head Start's funding was recently cut. So Ms. Barrett came to a somewhat controversial conclusion - the sector should stop killing itself over evaluation as a means to successfully fundraise. And she's right, but I don't agree in the way she meant it. In my opinion, the sector SHOULD NEVER have been killing itself over evaluation for funders. It should be killing itself over evaluation for its self - for its clients - for its communities - for its missions.

I am lucky to work at an organization that believes in excellent programming first and foremost. The idea is that money is attracted to excellent programs - and though there are times when intensive fundraising might get us more money, the focus on programs feels more true to our mission. And as I'm exploring ways to better evaluate at Volunteer Vancouver, let me assure you that it's not to attract money, though that would be nice side effect. It's to determine if the programs are impacting our client organizations in the ways that we hope. It's to determine if the resources we put into our programs are efficiently and effectively utilized. It's to take a look at our program mix and identify which are fulfilling our mission for the greatest number of clients. And then we can make those programs even better than they already are. And hopefully the money will come.

I hope the MSOs do collaborate to identify meaningful evaluation for our programs and services. If we are training others how to make their organizations strong, it shouldn't take too much for us to work together and make our own organizations strong, as the first examples for those very clients.

Comments

Anonymous's picture
Anonymous (not verified) on

Thank you for making this incredibly important point about evaluation: We should do it for the people who expect and deserve to get something meaningful out of programs, not for funders.

Head Start funding may have been cut despite good evaluation, which is unfortunate. Nevertheless, I think times are changing. Funders and donors are becoming more aware of the need to look at whether an organization is truly making a difference to the people it serves. Producing socially valuable outcomes is extremely difficult, and contrary to popular belief (which assumes that data is only for funders, because we know we're doing good), many many organizations do not succeed. Obama gets it; he really wants to fund programs that work.

Of course, rigorous external evaluation is expensive and not something all, or even most organizations should engage in. It is appropriate once an organization has successfully implemented a program model with consistency over time, and has good reason to believe that the program is working, among other conditions.

I'll be speaking at the Alliance conference tomorrow (Doing More Harm Than Good?) about what should be the starting point for all organizations: performance management. Performance management is internal, real-time data collection and management for the purpose of ensuring that programs are implemented as intended and ultimately making progress toward outcomes as expected.

That is something every organization can and should engage in. It is not cost-prohibitive, and it is necessary in order to manage programs well and possibly reach a point where an external evaluation is appropriate. Without performance management, programs can't know whether they are reaching the people they intend to serve, whether they are providing services at the frequency, duration and level of quality necessary for people to achieve outcomes, and whether they appear to be achieving outcomes. And producing meaningful outcomes, well, that's why we are involved in the nonprofit sector, in the end.

And by the way, the organizations I know that are most successful at raising funds these days are the ones that can show performance data (collected internally), and are willing to explain how they have learned and improved programs as a result. First Place for Youth will tell their story tomorrow.

Ingvild Bjornvold
Director of Advocacy
Social Solutions

Contact

info@thevantagepoint.ca
t: 604 875 9144
f: 604 875 0710
1183 Melville Street
Vancouver, BC V6E 2X5

Connect

Donate to Vantage Point and transform the leadership competencies of British Columbia not-for-profit executive directors, board members, staff and volunteers

Facebook icon.Twitter icon. Twitter icon. Youtube icon.

Subscribe

Accreditation