I'm your host David Wand and welcome to episode five, part one, where we introduce a new international development organization. That international development organization is the Mennonite Development Associates, located headquarters in Waterloo, Canada. You can learn more about the Mennonite Development Associates International Development Organization by going to their website at www.meda.org.
We're gonna be talking about one of the projects that they have been provided money from the government of Canada, courtesy of the Canadian taxpayer. This project cost the Canadian taxpayer $19,040,307. The title of this project is, A New Partnership for Sustainable Impact Investing in Frontier Markets. This project, unfortunately, doesn't specify the specific developing countries where these monies are spent.
Instead, it only indicates that 30% of the $19 million will be spent in Africa, 30% spent in the Americas, and the remaining 40% being spent in Asia. I have acquired, through an access to information request, the performance measurement framework, or PMF, for this project.
Reviewing the PMF, I have discovered there are nine outcomes that are expected to be achieved from this $19 million, and 23 outcome indicators are in the PMF that are being used to determine whether or not these nine outcomes have been achieved. Unfortunately, based on my review of all 23 outcome indicators, none of those indicators are able to adequately measure the achievement of the outcomes.
Therefore, the Mennonite Development Associates cannot make the claim that they are achieving these nine outcomes based on the performance measurement framework that they have designed. So in this part one, I will just give you a brief description of what the services are that this $19 million is being paid for, and it falls into three areas. The first area is funds that are provided to investment firms, and funds or grants that are provided to small, medium enterprises.
And a third component of this project is paying for Canadian firm managers, investment firm managers, to mentor investment firms in developing countries. That is, mentoring the managers of investment firms in developing countries. In those three areas, particularly investment funds and small, medium enterprises in these developing countries receive money.
They also receive, these investment firms and small, medium enterprises, technical support on social and environmental principles so that they can access these funds from this project. And as I mentioned, the investment firms in these development countries, their managers also receive mentoring from investment fund managers from Canada. The small, medium enterprises receive grants for social innovation. And they also receive technical support on social innovation.
So basically, those are what the funds, the $19 million are used for. And what I'm now going to do is give you two examples of outcomes from the performance measurement framework where the indicator, the outcome indicator that's been designed to determine whether or not the outcome has been achieved is flawed. So we'll start with this outcome from the performance measurement framework. The outcome is strengthened management capacity amongst investment fund managers in selected countries.
The indicator for measuring this strengthened management capacity is, and I quote, level of business skills and knowledge among mentored fund managers. Sounds pretty good. It sounds like they're ready to measure some skills and knowledge levels.
But unfortunately, when you look at the performance measurement framework, once again, it falls into the problem of self-reporting bias, where these mentored fund managers are themselves expected to self-assess their quote, acquisition of knowledge and skill, rather than being tested objectively on their knowledge and skill levels regarding how to work as a fund manager in terms of their management capacity. They need to be objectively tested on their knowledge and skill levels.
So that's the problem with that indicator. Even though it sounds good, it's only by getting a hold of the performance measurement framework that we see that once again, there's a bias and they're being asked to self-assess themselves rather than being tested objectively from somebody externally. The second outcome that I'm going to look at is called strengthened business practices for economic growth, job creation, and social impact amongst investment firms and small, medium enterprises.
The indicator to measure whether or not these strengthened business practices have been achieved is qualitative evidence of strengthened capacity of investment firms to promote improved social and environmental practices. The problem here again is self-reporting. These investment firms self-report their quote, strengthened capacity to promote through case studies and most significant change stories.
Again, it would be better to test people in these investment firms on their knowledge of social and environmental principles, plus their skill levels on how to promote these principles. Plus, a comparison group of people in investment firms outside of this $19 million project would be also helpful so that you can show that the $19 million project had people at these investment firms that you were providing them with training and technical support on these principles.
You would hope that they would have higher knowledge and skill levels on how to promote these social and environmental business practices compared to another group of people in investment firms that don't receive this project training. So in part two, we're going to invite the Mennonite Development Associates to respond to our critique of all 23 outcome indicators and why they're flawed.
And we'll also have another evaluation expert on part two of the podcast who will work with me in going through at least nine of those outcome indicators to make sure that we cover all nine outcomes of the project. Hopefully we'll go through more than just nine outcome indicators and hopefully cover all 23 outcome indicators to explain how each one fails to properly measure the outcome.
So stay tuned for part two where we will go through in more detail all 23 outcome indicators as to why they have a problem with not properly measuring the outcomes. So if you're interested in receiving the performance measurement framework for this project and also the outcome indicator summary for all 23 outcome indicators, explaining which of the five problem areas each outcome indicator falls into, feel free to send me an email at evaluatecanadaaid at gmail.com.
And if there's any evaluation experts out there who would be interested in being on a part two of the podcast where we go into depth on each outcome indicator, feel free to email me and I will send you the remaining performance measurement frameworks that I have for season one and you can choose one and then I can also send you the notes that I have from my perspective of why the outcome indicators are flawed or not.
I look forward to doing part two and stay tuned for part two which will be coming up very soon in a couple of weeks. Bye for now.