The MCF, the Freemason’s Charity, is a young foundation formed in 2016 to provide support to charities and individuals in need and their families. From its inception to August 2023, the MCF has provided around £140 million in support, on average £20M a year.
In recent years, impact and evaluation have gained increased importance for the MCF. Our new strategy places a greater emphasis on ensuring we have a positive impact on those we support, for which evaluation practices and the generation and use of impact insights now underpin our evolving approach. In March 2022, the Impact and Evaluation team was established to spearhead this transformative endeavour.
Commission or not to commission evaluation
Literature suggests successful commissioning evaluation hinges on strategic decision-making and that there are also dichotomies of commissioning that need to be weighed. Commissioning might not always be straightforward, and selecting the right path can be critical, as missed steps can lead to less than desirable outcomes, as sometimes it can go wrong. How then does the Impact and Evaluation team at MCF decide whether to commission an evaluation? Here we delve into some of the considerations that guide our decision-making process. They are not an exhaustive list but rather what I believe are the active ingredients in a recipe for success.
1. Purpose of your evaluation: a crucial ingredient
One of the primary considerations when commissioning an evaluation is understanding its purpose. I identify two broad types of purposes to consider for your evaluation:
Strategic implication. This involves considering the big picture and contextual understanding of the organisation’s current state. How are decisions made in your organisation? What are the most influential channels in your organisation? Where does the creation of external evidence sit in creating or influencing internal change? Does the evaluation visibly align to the organisations’ needs? If not, do you still want to pursue this work? These have been key questions I have considered when commissioning evaluation at a time of organisational impact and evaluation transformation.
Defining purpose and deliverables of your evaluation. Although it is always important to adopt a structured approach to impact and evaluation projects generally, whether they are internally or externally commissioned, this is particularly important when commissioning work externally because it can have legal implications down the line. Providing specific examples, such as tables and graphs, for a statistical analysis I commissioned, may seem prescriptive, but it allowed the Data Analyst to focus their time on the task at hand and even to provide additional data analyses and visualisations. Ultimately, clear deliverables translated into enhanced value for money and high-quality outcomes.
2. Leveraging internal delivery resources
Consider the limitations of your team resources. I mentioned earlier that impact and evaluation practice in its current approach is new to the MCF. As the I&E team is busy embedding our new approach, establishing processes, conducting data collection and analysis, and generally building the organisation’s impact and evaluation capability and activities to drive a positive data impact culture, the commissioning of evaluation by other experts has been used to accelerate resource creation to underpin some of the team’s activities. For example, the insights we gathered from the statistical analysis mentioned above were used to understand one of our programmes from a different impact
perspective for the first time as it answered questions new to us, such as the distribution of Index of Multiple Deprivation (IMDs) amongst individual beneficiaries. In turn, some of the newly generated insights were published in the 2021/22 MCF Impact Report, fostering internal discussions around reshaping our approach to evidence generation and utilisation. Such activities would have been difficult for the Impact and Evaluation team to deliver because of the team’s time capacity: this highlights the value of tapping into external expert skill sets, leading to greater effectiveness and efficiency of resource allocation.
3. Selecting the right evaluation partner
My final advice is about making the right selection of your evaluation delivery partner. I always encourage you to allocate sufficient time for the appointment of the delivery partner. If the outcomes of the work will also impact other teams, it is advisable to work with them in the selection of the consultant too. By involving them, you will increase the chance that the outcome of the evaluation will have a more extensive influence across your organisation. Regardless of your method of appointment, it is paramount that you get a good sense of whether your working styles match, how each of you might respond to contingency if things don’t work as expected, and how well you communicate effectively with one another. It is okay to be detailed as it helps to set expectations for
all parties involved. As I once read “not only agree on a value, also agree on processes”; a principle I have stuck to ever since.
In brief, the commissioning of evaluation should be a carefully considered one, influenced by a blend of strategic insight, clear purpose, internal resource assessment and a sensible partner selection. By following this recipe, I aim to craft a cocktail of fruitful results that ultimately aim to generate a greater impact on those we support.
Jo is on the panel for our upcoming event: To Commission or Not to Commission? on 01/11/2023. Find out more about this event and book your place today.