Welcome to the Improving Development Evaluation Podcast. I'm your host, David Wand, and in this episode, we're going to feature the international development organization CUSO International, that's C -U -S -O International. You can learn more about CUSO International at their website at www .cusointernational .org. We used to affectionately refer to CUSO back in the 1980s when I was in
Seattle. leone with cuso as the canadian unemployed socialists overseas and the david haslett wonderful man who was the field staff officer for sierra leone back then he was the one who introduced me to that acronym but of course we've made some other ones that are quite funny but we'll stop right there and get on with the episode we're going to be looking at two cuso projects one called blueprints at a cost to the taxpayer of
$2 .5 million. And another project called Building Bricks, at a cost of $5 million to the Canadian
taxpayer, for a total of $7 .5 million. And Blueprints has nine outcomes, and we're going to be looking at the performance measurement framework and seeing how well the 13 outcome indicators for those nine outcomes properly validly measure the achievement of those outcomes and for building bricks we're looking at eight outcomes for that project and a total of 15 outcome indicators designed to measure whether or not those eight
outcomes have indeed been achieved. Before we get into it I just wanted to give you some background. There's been some significant delays in producing these performance measurement frameworks. Rebel Media, they're not very well respected in Canada but they've reported in their opinion, that some of these access to information requests like mine for PMFs are intentionally being delayed because, of course, that information that they
release makes the government look bad. Now, I can only speculate as to why there have been delays in me receiving my performance measurement frameworks. These performance measurement frameworks need to be produced before the government of Canada can disperse funds to CUSO International, for example. I requested these two PMFs for these two projects back in October of 2024. It's already been over a year since I received them. So I
suspect there's a bit of delay going on. And I've also requested two other international development organizations and their PMFs. And one of those organizations, the PMFs I'm asking for... The projects cost over $50 million, and I'm very upset. It's been over a year, and they keep delaying the release of these PMFs. There's no excuse for it because they've already been produced.
They're only 10 to 15 pages long, and they have to be produced by the International Development Organization before they disperse the funds. And as we speak, funds are already being dispersed for these projects. because I can see it on the Government of Canada Project Browser website, and I'm still not getting the PMFs. So that's a bit disturbing. If any of you want the PMFs and my critique of the PMFs for each of these episodes, you can email me at evaluatecanadaid
at gmail .com. And as you know... The link for this episode is sent to the Secretary of State for International Development, along with the shadow critics at the Parliament in Canada. So everybody knows, regardless of political affiliation, that your tax dollars are not being properly evaluated when it comes to foreign aid projects. So you can also email in the episode. I'll provide
the emails to recommend. To the Secretary of State, feel free to email them yourself with the link for this episode telling them to defund these international development organizations until they start producing proper outcome indicators for the outcomes that they keep claiming falsely on their websites that they're achieving when we don't even know that they're being achieved. So that's important. So I'll also be producing a petition to... at change .org that I'll be
putting in each of the episodes notes. So if you want to sign a petition urging the Secretary of State for International Development to indeed urge them to defund these organizations until they produce proper outcome indicators and also ideally we want these performance measurement frameworks as well as the data that goes with them. to be produced on the Government of Canada Project Browser website, as well as the International
Development Organization's websites. Because currently, they produce a lot of data, but it's useless. Garbage in, garbage out. And that's the problem. We've discovered that the design of these indicators in the first place is flawed. And in some cases, as I've noted in other episodes, they don't have any outcome indicators at all. And they keep claiming that they're achieving outcomes. But if you look at the reports they send me, there's no outcome indicators at all.
Instead, they just talk about outputs, how much services they've delivered, but no evaluation of what happens after those services are delivered. And like I said, if you believe in foreign aid and you want it to continue to be supported without somebody coming along and saying, that there's no evaluation and therefore we're going to scrap them, these foreign aid projects. It's better if you believe in foreign aid to show that the
foreign aid is actually working. It's effective and it has impact, which is what the development assistance criteria is all about. But as you've discovered, probably through my podcast and other information, this is not happening. So it's a direct threat to funding, for foreign aid projects. So in many cases, it's quite simple to show that foreign aid is working. But in some cases, it's a little bit more difficult, but there should at least be an attempt to measure and then evaluate.
But that's not going on. So I'll leave it at that and let's get into it. So blueprints, nine outcomes, 13 outcome indicators. Give you a brief description. of what the project does. It's focused on the following countries in the Caribbean, Dominica, Grenada, St. Lucia, and St. Vincent. And they have two target groups, public institutions in those countries, as well as women's rights organizations. And they focus on training these public institutions and women's rights organizations
on gender equality. By accessing materials on online platforms to address issues of gender equality. And CUSO helps them organize meetings to share best practices on addressing gender equality issues amongst public institution representatives and these women rights organization representatives to increase awareness among these two target groups on how to address gender equality issues.
So they also spend money, CUSO, this funding, $100 ,000 to increase engagement between these public institutions and the women's rights organization that represent girls and women, which leads to these public institutions being more effective in addressing these gender equality issues. So, for example, women and girls will show up to attend these dialogue spaces between the girls and women and the government, public institution.
This includes communication campaigns and communication engagement sessions to increase access to public services. So these girls and women will show up, provide feedback through scorecards to government reps. The government feedback committees will be set up to receive these scorecards. And the government staff in these public institutions are trained in, quote, accountability practices. They deliver information fairs on gender services
to women and girls. They're trained in these public institution representatives on gender data and how to incorporate these gender development indicators into their departments. They're trained on gender responsive operational plans, including how to reduce barriers to access services for girls and women. They're also trained on gender responsive M &E, monitoring and evaluation and service delivery. They're also trained the public institutions on gender budgeting and how to adapt
gender budgeting tools. Workshops are delivered. Gender equality CUSO volunteers are also deployed to the Caribbean. They're also trained these government departments on how to develop gender responsive capacity development plans. They're also trained on how to complete needs assessments to identify technical assistance needs for gender equality. So government staff are reporting increased knowledge and skill to carry out these needs assessments. So that gives you basically an idea
of the Blueprints project. Now moving on to the Building Bricks CUSO project. It's going on in also the Caribbean, Belize, Dominica, Grenada, Guyana, Jamaica, St. Lucia, St. Vincent, Grenadines, and Suriname. And their goal is to influence legislative reform and increase access to services for those from the LGBTQIA plus lesbian, gay, bisexual, trans, gendered, queer community through organizations that represent those communities.
What they often focus on is advocacy, training on how to design and deliver communication strategies. Also training on how to develop awareness campaigns, advocacy agendas, and how to sensitize and train service providers on the rights of the LGBTQIA plus community. And also training these organizations that represent the LGBTQIA plus communities in resource mobilization and monitoring and evaluation
system design. Also training them on governance, leadership, ability to complete needs assessments, how to respond to the needs of these particular communities for both the organizations that represent them and the service providers. So the key target groups there are the lesbian, gay, bisexual, transgender, queer, IA plus organizations, as well as the service providers themselves. And the cost of that is building bricks is $5 million.
And we have eight outcomes for building bricks and 15 outcome indicators we're going to look at from the performance measurement framework for those eight outcomes. So there you have a
broad description of the two projects. And before we get into the PMF for both of them, to see if they actually did properly measure and evaluate the achievement of those outcomes for those projects, we again will talk about whether you are living in the Global South, where it's usually, but not always, hot and humid, or you work for CUSO International or for another international development organization. And lo and behold, you may have to go to the global self to actually monitor
and evaluate one of your projects. And so it's hot and humid there. So you probably want to check out some sun hats, sunglasses, and some cool clothing. And if you do that, you'll see in the episode, I have a link called https colon backslash backslash mydeals .page backslash one h. And if you click on that link, you will get access to these three companies where you can get sun hats at ericjavits .com, where you can travel friendly and protectively from the sun
with these hats. You get a 20 % discount off any product if you use my link, mydeals .page. For some reason, if you can't, you always use wand cool t -shirts, W -A -N -D -K -O -O -L t -shirts. But the link should work. And you'll get a 20 % discount. And I get a 20 % commission from the total purchase you make. So you might be wondering these sun hats at ericjavits .com. How protective are they from the sun? You get ultraviolet protection factor with a 50 plus
ultra protection. factor rating, blocking 98 % of UVA and UVB radiation. They're travel -friendly, easy to pack, but retains its shape when you visit your international development project located in the Global South. You can also pick up some sunglasses at akila .la, A -K -I -L -A dot L -A. Includes prescription lenses, all made from 50 % recycled material. You get a 10 % discount if you use my link, mydeals .page backslash 1HJL
that you'll see in the episode notes. And I get a 15 % commission when you purchase any of the products using my promo code or that link at mydeals. So you have your hat and your sunglasses to protect you from the sun and the global south, but you need the right clothes to be comfortable in that hot. and humid climate. How about some men's polo t -shirts or button -ups at hypernaturalstyle .com. You get 20 % off any product when you use my promo code JuanCoolT -shirts or the link.
And I get a 20 % commission of the total purchase you make when you use my link at my deals. So what is cool about these hypernatural t -shirts? They have a patent -pending hyper -cooled jade technology that, when it's hot, lowers your body temperature by 3 to 5 degrees. They also are breathable, thanks to Supina cotton. And they are sold in premium retailers like Nordstrom. And they were voted the best men's polo shirts in 2024 in Men's Journal, Forbes, and Esquire.
So, that's basically it there. So let's start with blueprints for gender equality at a cost of $2 .5 million to the Canadian taxpayer. And we'll take a look at some of the outcomes. Nine outcomes, 13 outcome indicators. The first outcome is improved capacity of public institutions.
in targeted Caribbean countries to share good practices and lessons learned on addressing the most pressing gender equality issues faced by the poorest, most vulnerable women and girls in all their diversity, including women's rights organizations. The first outcome indicator for that outcome is percent of ministries, departments, and agencies that have improved their capacity to coordinate with public institutions across
the Caribbean. to share good practices and lessons learned, to address most pressing gender equality issues faced by poorest, most vulnerable women and girls, disaggregated by institution, sector, and geographic location. But if you look at the performance measurement framework, you'll find there's a conflict of interest, as CUSO and the government departments do a, quote, partnership
review. So there's a self -reporting bias. Instead, what they should be doing is using an external evaluator should be measuring the technical capacity of the individuals inside of these public institutions to see if their capacity to share best practices on gender equality have actually been done. So that's not a very good indicator. They can't claim, once again, they've improved the capacity of these public institutions. because they have
a conflict of interest. They need an external consultant to come in and use an objective measure instead. The next outcome they have is enhanced capacity of public institutions to engage with the poorest, most vulnerable women and girls in all their diversity, including women's rights organizations in targeted Caribbean countries.
The outcome indicator for that outcome is... percent of ministries, departments, and agencies that have increased their knowledge about barriers faced by poorest, most vulnerable women and girls to increase engagement with poorest, most vulnerable women and girls, disaggregated by institution, sector, and geographic location. Again, if you look at the performance measurement framework, who's doing this? It's a conflict of interest as CUSO and the government departments do a quote
partnership review. Instead, an external evaluator should be measuring the knowledge levels of the individuals inside of these public institutions to see if their knowledge levels of recalling and identifying to increasing engagement with vulnerable women has actually occurred. So again, this is not a good outcome indicator to claim that they're achieving, quote, enhanced capacity
of public institutions. Doesn't work. The next outcome they've got here is increased knowledge and skills of public institutions to effectively deliver. gender responsive public programs and services that address the most pressing gender equality issues faced by the poorest, most vulnerable women and girls in all their diversity in targeted
Caribbean countries. The indicator for that is percent of ministries, departments and agencies who have improved their performance in delivery of open inclusive and gender responsive public programs and services for poorest most vulnerable women and girls disaggregated by institution sector and geographic location yay in this one they get a green light they actually use and mention external consultant which is good And there are data sources in the PMF, document review,
case studies, that suggest this can be done objectively and free from the public institutions self -reporting on their skill levels in delivering gender responsive services, which is biased and not a proper valid measure of this outcome. But in this case, they use an external consultant. So it begs the question, are they going to use an external consultant again? for some of the other outcome indicators that are coming up. Let's see. So, that's good.
The next indicator, same outcome, increased knowledge and skills of public institutions, blah, blah, blah. The next indicator they've used for this same outcome is percent of ministries, departments, and agencies that have adopted results -based gender -responsive planning, monitoring and evaluation procedures and practices. disaggregated by institution, sector, and geographic location. Did they use an external consultant again for this? Sounds like they should. No, they didn't. What did they
do? They dropped the external consultant here and instead are using CUSO, which is a conflict of interest. They should use the same external consultant to go in and check to see, maybe using a checklist. If the public institutions have indeed adopted monitoring and evaluation procedures. This in turn can then be used to see if they're able to achieve the outcome of skills to evaluate whether they have effectively delivered gender
responsive services. Clear self -reporting bias and strange that they will not use the same external consultant. So that's not good. So far, only one good indicator. Now, again, New outcome here, enhanced gender responsive budgeting processes of public institutions to address the most pressing gender equality issues faced by the poorest, most vulnerable women and girls in all their
diversity in targeted Caribbean countries. The outcome indicator they use for this outcome is percent of ministries, departments and agencies with increased knowledge and skills on gender and participatory. budgeting processes disaggregated by geographic location and sector of ministries, departments, and agencies. Great indicator, but
no external consultant. Instead, again, CUSU is doing it, and even worse, they are asking the individuals in these public institutions to self -report on their knowledge levels in the target column of the performance measurement framework, and even ask them how quote satisfied they are with those knowledge and skill levels as noted in the data source of the performance measurement framework where it shows a satisfaction survey a clear again self -reporting bias instead
they need to get that external consultant to develop some objective criteria of what one needs to see to conclude that the public institution does indeed have, quote, a gender responsive budgeting process in place. So again, they've done it once, an external consultant, but somehow they don't want to do it for the other indicators.
Not good. The next outcome is increased knowledge and technical skills of public institutions to develop and implement gender responsive public policies, programs, and services that address the most pressing, Gender equality issues faced by the poorest, most vulnerable women and girls in all their diversity in targeted Caribbean countries. Now for this, they have three indicators
for this one outcome. First indicator is average, open, inclusive, gender responsive and accountable score, public policies, programs and services. Design implementation for poorest, most vulnerable women and girls component. The other indicator they have for the same outcome is average open, inclusive, gender responsive and accountable participatory needs assessment score, gender
equality component. And the third indicator for this is an average open, inclusive, gender responsive and accountable score for monitoring and evaluation and learning component to collect gender disaggregated data. So the three indicators for this results in a, quote, needs assessment report completed by, and quote, if you look at the PMF, joint sessions between CUSO and the individuals within
these public institutions. Once again, they need to get an external consultant to measure their technical skill on the three areas that would lead one to conclude. The skill on, quote, developing and implementing gender responsive policies, programs and services has indeed increased. Right now, we have a conflict of interest where there's no external consultant. And CUSO is doing this internally with the public institutions that they're trying to measure and evaluate. Not good.
And also a needs assessment report. doesn't seem to be a valid measure of increased skill on, quote, developing and implementing gender responsive policies, programs, and services. So that's the problem there. Moving along to intermediate outcomes, we have enhanced gender responsive public policies, programs, and services to address the most pressing gender equality issues faced by the poorest, most vulnerable women and girls in all their
diversity. in targeted Caribbean countries and the outcome indicator for that is average ministries departments and agencies open inclusive and gender responsive participatory needs assessment score so in the performance measurement framework the data collection methods are between CUSO and the public institutions producing a needs assessment report this has nothing to do with measuring whether the services they delivered were gender
responsive. You need to ask the women and girls who received the services from these public institutions if the services were gender responsive. So again, a lousy indicator that doesn't measure at all enhanced gender responsive services. The next outcome is increased effective delivery of gender responsive services. that address the most pressing gender equality issues faced by the poorest, most vulnerable women and girls in all their
diversity in targeted Caribbean countries. The outcome indicator from the PMF is percent of poorest, most vulnerable women and girls with effective access to open, inclusive, gender responsive and accountable public programs and services in four countries. Disaggregated by socioeconomic status, gender. age, ability, ethnicity, language, geographic location, and type of service. Sounds great. Problem is when you look at the PMF, the
data sources are CUSO blueprint reports. Doesn't the access come from the locations where these women and girls show up to receive the services? So I guess we are relying on these blueprint reports that contain the percent of women and girls that receive the gender responsive services.
To show that this access has increased, a better word would be maybe effective, we need to see the measuring instrument as well as the criterion used to conclude that the services delivered indeed meet the standard of being described as
being gender responsive. Again, they should use an external consultant reviewing the records where the service was delivered rather than using a CUSO blueprint report because that would have no self -reporting bias so that's the problem there is again they're relying on internal QSO reports when if you really want to see if these girls and women have had effective access to gender responsive services you would want to have the external consultant go in and review
the reports That would be a better approach. The next outcome is actually for the same indicator, excuse me, for the same outcome, increased effective delivery of gender responsive services, was level of satisfaction of poorest, most vulnerable women and girls in all their diversity with open, inclusive, gender responsive and accountable public programs and services in four countries. This is great. Great indicator. They're actually going to go to the women and girls and ask them. Are you
satisfied with the services? And are they indeed, in your opinion, gender responsive? We'll give you a definition of what we mean by that. Fantastic. Problem again is that CUSO and the public institutions, not an external consultant, are responsible for asking these women and girls how satisfied are they with the services they receive from these public institutions. Now, it's funny. They used an external consultant before, as I mentioned.
So why not for this very good indicator? A good indicator ruined by CUSO public institution conflict of interest. Get an external consultant to complete the satisfaction survey and produce the report that claims, and I quote, increased effective delivery of gender responsive services. The current blueprint reports as the data source in the PMF
are potentially biased. The next outcome statement improved collaboration and dialogue across public institutions in targeted Caribbean countries on the most pressing gender equality issues faced by the poorest, most vulnerable women and girls in all their diversity. The outcome indicator for that outcome? Percent of staff who recognize public institutions as key actors in sharing knowledge to address most pressing gender equality issues. faced by poorest, most vulnerable women
and girls. So they've got increased recognition. Recognize. So increased recognition, knowledge. It does not equate with, quote, the outcome, improved collaboration and dialogue, which is a behavior, right? They need a better indicator for increased collaboration and dialogue. such as either frequency of meetings and communications specific to actions to address gender equality. Note, this is still process rather than results, where process is not measuring any changes in
increased gender equality. So that's an example where they've got an indicator, but it's not related to measuring the outcome of improved collaboration and dialogue. Then finally, the ultimate outcome in this project is enhanced gender responsive governance in Dominica, Grenada, St. Lucian and St. Vincent and the Grenadines for the poorest, most vulnerable women and girls in all their diversity. The outcome indicator for this ultimate outcome is number of poorest,
most vulnerable women and girls reached. by open, inclusive, and gender -responsive public programs and services in four countries. Now, this is a bit of a problem because how is this any different than the indicator for 1200, which was percent of women and girls with effective access to gender -responsive services? It looks like they're going
countrywide here. It would be better to actually use a measure of gender equality that is an expected outcome if there is an increase in gender -responsive services delivered. Where more women and girls get those gender responsive services because they've already measured that in some of the other outcome indicators that I just mentioned. So the next level would be at the country level is governance regarding gender responsiveness and gender equality being achieved for women
outside of the project, right? So an example could be the UNDP. Gender inequality index, where they look at maternal mortality, adolescent birth rate, females that have secondary education, the female share of parliamentary seats, the female labor force participation rate. Now, although attributing the project to those changes in the gender inequality index would be difficult, at least it should be compared with the project measure. of increased access to gender responsive
services. Because I think in the outcome, the ultimate outcome, they're implying we're going to go to the country level outside of the project. In summary, for this project, you've really got 13 outcome indicators, but all of them are bad except one. There's only one really good indicator. There's just too much self -reporting bias going on. And in some cases, the indicator is not even related to the achievement of the outcome. So that's the big problem with this performance
measurement framework. So I could, again, would conclude that they're not achieving any of their outcomes because again, they're not properly measuring them. So they can't make the claim that their outcomes are being achieved. So let's look at Building Brooks for $5 million cost to the Canadian taxpayer. They've got eight outcomes and For those outcomes, they've got 15 outcome
indicators. So if we start from the bottom with the immediate outcomes that they expect to achieve, we've got enhanced coordination and collaboration action on rights -based agendas among local and regional LGBTQIA plus organizations and networks in the Caribbean. The outcome indicator for that was number of organizations and networks coordinating human rights and gender equality agendas. Now, you're probably suspecting already, it's not
directly related to the outcome. How does increase in the number of organizations equate with, quote, enhanced coordination? Even if the increase in number of organization happens, was it due to the project output? Or could they increase that number by themselves without you spending $5 million? The project creates, quote, dialogue spaces that are then used to do this coordination. But could this happen without the creation of
these dialogue spaces? And if so, would this increase in the number of organizations doing this coordination, would it be much greater than without these project dialogue spaces? So this is a case where... Yes, they could maybe claim increased coordination and enhanced coordination, but maybe it would happen anyways without the project. So this is where they're not really looking at impact. So it's a little questionable, right? So the next outcome is, again, enhanced
coordination. And they have another indicator for it. And they have number of organizations and networks reporting increased ability. to participate in legislative reform in the Caribbean. Again, self -reporting bias, right? If you want to see if this project has actually increased their ability, their skills and knowledge to engage in trying to get legislative reform, then don't ask them themselves, hey, how you feel?
Don't. That's self -reporting bias. You need to get an external consultant to objectively measure. whether the individuals within these organizations have indeed increased their ability to participate in legislative reform. So you've got self -reporting bias. The next outcome statement, increased capacity of local and regional organizations and networks to advocate and campaign for LGBTQIA plus rights and gender equality using an intersectional
approach. The outcome indicator here is number of individuals reached through national and regional campaigns. The survey that they use here in the PMF on reach is excellent. So it's very good. That indicator is good. The problem is the other indicators that they use for the same outcome. Number of large -scale national regional campaigns
designed and launched. So that's good too. That's the second indicator that's good because as the number of launches goes up, that shows they've increased their capacity to advocate and campaign for LGBTQIA plus rights. So these are two indicators that are actually quite good. So that's well noted. But the third indicator, once again, is biased. And that is... Number of these organizations that report increased ability to advocate. I would just take it out. This is self -reporting
bias. Getting the number launched is good. Getting the number that have been reached is good. And that by themselves, those two indicators are good to show that they've actually increased their capacity to advocate and campaign for those rights. That's a good example. So not bad there. Next outcome, increased access by LGBTQIA plus people to safe, inclusive, environmentally friendly, and gender responsive socioeconomic services
in targeted countries. The problem here again is the outcome indicator is the number of organizations and networks that report increased ability to coordinate with and strengthen the ability of service providers to deliver quality services. Self -reporting, again, it is biased. So this should be taken out. But good news, if we look at the next indicator for increased access, they actually go to the people in those communities, LGBTQIA +, and ask them directly the number that
have reported increased access to services. Very good indicator. of the outcome if we mean improved or adequate quality in achieving gender responsive services is equivalent to increased access we have to be careful here just as the number goes up that are reporting they got the right services we should also make sure that the quality is adequate too but we're going to assume that for now and it is in the pmf The surveys seem to be doing that. So that's another measure. That's
three good indicators now. So that's good. But again, we get back to problem of self -reporting with this next outcome. Increase knowledge and skills of local and regional organization and networks in governance, leadership, and management to represent LGBTQ. plus rights in targeted countries. So as you remember from this project, there's lots of training going on for these individuals. So what they have to do is stop asking them how
they feel. Stop asking them to self -report on whether or not their skills have been increased. It's getting ridiculous. It just doesn't look good. What they need to do is get an external measure of those skills in leadership and management. Get a consultant to come in. And say, sorry, I'm going to measure whether or not you've learned what you're supposed to learn. I'm going to test you. Very easy to do. Self -reporting, bias,
not good. And they also do it again for another indicator on human resources governance and monitoring and evaluation. Self -reporting, where the indicator is percent of staff of organizations. Reporting increased knowledge and abilities in human resources governance and monitoring and evaluation. Again, they're using self -reporting. They need to use an external measure of increased knowledge of human resources governance and monitoring and evaluation. Next indicator after these immediate
incomes is the intermediate outcomes. Here's the outcome statement. Increased effectiveness of local and regional organizations and networks to coordinate the LGBTQ. IA plus rights movement in the Caribbean. The outcome indicator is number of organizations and networks, again, reporting increased effectiveness to implement mechanisms that positively influence gender sensitive policies. Again, a bias. Don't ask them. Self -reporting.
Get an external measure of increased effectiveness to implement mechanisms that influence gender sensitive policies instead. This could complement the next measure on effectiveness to coordinate. They should also add what exactly would a gender -sensitive policy look like in terms of legislation. So that could be aimed for. Maybe that comes later. So another indicator for increased effectiveness is number of policies, laws, and frameworks presented for legislative reform by... organizations, and
networks. This is a great measure of increased effectiveness to coordinate. But be aware, just because they've increased the number of policies presented, they still could be garbage, right? You need someone to look at the quality of these policies, laws, and frameworks. Is it sufficient as reviewed by external experts or even by the government agencies that have received this advocacy document? or recommended policy. It doesn't matter.
It could be an external expert. It could be the civil servant that's received the document and says, this is crap. It doesn't even include any data to support the change in legislation and the need for this change. So not just the number presented, but at least they can claim that they're doing it. But increased effectiveness, you need to look at the quality and even better. to see if it made a difference indeed in any changes
in legislation. The next indicator is enhanced performance of local and regional organizations and networks, gender responsive programming and advocacy for rights of LGBTQIA plus people and gender equality in targeted countries in the Caribbean. The outcome indicator there, again, Self -reporting bias, number of organizations and networks that report increased ability to advocate for the rights of LGBTQIA plus people.
So again, a problem with self -reporting. Instead, they need an external measure, maybe an external consultant of enhanced performance instead. If their performance was convincing and enhanced, you would expect a press release or even better. legislation indicating a change, reflecting the changes that were advocated for in that, quote, enhanced performance. Another indicator for enhanced performance is number of organizations and networks
with increased delivery of programming. So they're asking for increases in programming in this advocacy. That's what they want, right? Increased delivery of programming. So why not just look at that? Forget about the number of organizations that are doing it. Under the target column, they have the number of organizations that have improved their delivery of services. But they want more
services also, not just improved services. Instead, list the new programs and what they were delivering and the number of people receiving the program by type of gender identity to see if it has increased. due to this project's training on advocacy and lobbying for increased access to services. Have the number of people receiving services increased is what should be measured here as the expected outcome. So here the indicator is not relevant to measuring the outcome of enhanced performance
to advocate for rights, right? You want to go to the end and see. If there's been an increase in the number of services delivered and also perhaps also legislative reform, because usually they go hand in hand. Another outcome here is improved institutional capacity and sustainability of local and regional organizations and networks in targeted countries in the Caribbean. The outcome indicator for that outcome, number of organizations using sustainability, best practices. Now, when
you read that, that's pretty good. But again, if you don't have the performance measurement framework in front of you, there's a little trick they're playing here. The outcome and its indicator that I just stated are very good, very clear. But if you look at the baseline data column in the performance measurement framework, it's not related to the indicator. Instead of going into an organization to see if they're actually, quote,
using sustainability best practices. They are just checking to see if the organization has a, quote, sustainability plan. That is not actually a practice or a behavior. It's just a piece of paper, a plan. Are they actually, you know, maybe in the budget? And for example, I'll describe. The target column only counts how many organizations were trained in sustainability. That is not relevant to the outcome statement. The target column does also have a measure of the number of organizations,
quote, implementing best practices. Let's hope there is a manual on the criteria on how to decide that an organization is indeed implementing sustainability. What is that sustainability? Well, one example, a reduction in foreign funds and an increase in developing country funds to become less dependent.
on foreign aid, i .e. more sustainable. So what you could do is actually show as a better indicator the budget of the organization and seeing a reduction in the foreign funding from CUSO or anybody else and an increase in the local fundraising where they've got money either from the private sector or and or from the government, their local government,
right? So that's an example where the indicator Sounds good, but when you look in the PMF, the data that they're actually collecting is not related to measuring the achievement of the outcome or even practicing the outcome indicator. The next outcome is increased realization of human rights by LGBTQIA plus people and advancement of gender equality in. Belize, Dominica, Grenada, Guyana, Jamaica, St. Louis, St. Vincent, and the Grenadines, and Suriname in the Caribbean.
The outcome indicator is number of LGBTQIA people who have realized their human rights. Now, if you look at the data source in the performance measurement framework, it indicates, quote, project reports along with a baseline total of 847. Now, this suggests only those in the CUSO project are being surveyed. So there is a bias. This is the ultimate outcome. And you would expect it to increase the realization of human rights
to go beyond the CUSO project people. So the survey needs to be countrywide to claim that their ultimate outcome of increased realization by LGB. LGBTQIA plus people has been reported
at the country level. So here's an example where I suspect, but it's not clear, but I'm assuming it is because they talk about ultimate outcome being at the country level because they list the countries so that they would need to go beyond just surveying the CUSO project people that received this advocacy or... the benefits of the advocacy
project. So thank you for listening and we will send this off and invite CUSO to respond to my critique for these two projects and feel free to email the minister, the Secretary of State responsible for him to stop funding CUSO immediately until they start developing better outcomes and, for example, getting rid of this self -reporting bias. Thank you for listening. And stay tuned when I get another PMF or collection of PMS. We will do another episode. Bye for now.