At our first ‘Bitesize’ event in February the evaluation team from Scope shared their experience of taking a co-production approach. The aim of Bitesize events is to focus on different elements of equitable evaluation, from the questions we ask to methodologies we use and how we disseminate findings .
In the spirit of co-production, the team have co-written a blog to share some additional detail about how they organised the team, the benefits of a diverse team, how they ensured reciprocity, the challenges they faced and their approach to dissemination.
Read on to find out more!
Thank you to all the ChEW members who came to our BITESIZE talk in February on how Scope has applied a co-production approach to evaluation. We really enjoyed presenting.
On the day we received some questions that we didn’t quite have time to answer. In this post the co-evaluation team from the project will talk around some of the queries we received. If you still have questions, please take a look at our published report or contact Jess, Scope’s Impact and Evaluation Lead at jessica.bricknell@scope.org.uk.
How the team was organised
There were six of us in the co-evaluation team. One person (Mustak, our Customer Representative) had used the Support to Work service before. The Co-Evaluators and Peer Reviewers were recruited using role profiles. These specified the sorts of interests and experience that the in-house team felt were necessary for the project. The profiles had fairly loose requirements but made clear we wanted to create a team with lived experience of disability because of the perspective and understanding this would bring to the evaluation.
We also paid an expert from an independent Disabled People’s Organisation to sit on our stakeholder panel.
Through this process we ended up with a diverse team with varied life experiences. Some team members had accessed employment support services outside of Scope. This brought helpful context to evaluating the service.
The benefits of a diverse team
The team’s diversity really added value to our work. It meant we could generate more ideas for methods and interpretations. We also think the way the team was set up gave evaluation participants confidence that their experiences would be truly listened to and properly represented in our research. We received feedback that the involvement of evaluators with lived experience of disability was a motivating factor for some participants to take part in the project.
The whole team were focussed on creating an accessible and open atmosphere for participants. We heard from both satisfied and dissatisfied customers.
The co-evaluation team also added a different dimension to key informant interviews and focus groups with service staff. Service advisers were able to direct their feedback to the Co-Evaluators rather than full time in-house colleagues.
“It was […] crucial for us to enable and support the service advisers to express their views candidly. We felt that enabling service users and providers to express robust and honest feedback was essential to help shape this evaluation.”
– Julie, Peer Reviewer
One of our Co-Evaluators, Amy, also had quantitative research experience. She was able to influence and direct some statistical analysis that our Data Analysis Manager was conducting on Support to Work.
Reciprocity
Rewarding everyone for their input is one of the fundamental principles of effective co-production. The project required substantial groundwork and thought about how to do this fairly. As project lead, Jess spent a lot of time ironing out finance- and HR-related questions about the team’s structure. Ultimately, some of the co-evaluation team were directly employed by Scope, while others received vouchers to recognise their contributions to project activities. If co-production is a new approach where you work, we’d recommend allocating project time to discussing what payment methods will work best for your organisation, project, and team members to ensure everyone is fairly compensated.
Now that Scope has formally adopted co-production as a strategic operating aim (see below), our wider Strategy and Impact team are working to develop a consistent remuneration policy for all co-produced work.
Challenges
We faced a few challenges along the way:
·Getting to know each other when working remotely: it took a while for the team to feel comfortable enough to collaborate openly
Getting used to new digital tools: this was challenging without in-person training or dedicated resources
Maintaining momentum and connection throughout the project: some activities and processes took longer than expected. This meant some team members went through periods of not actively contributing
Managing time and flexibility: We also didn’t complete the project within our expected timeframe. From planning to dissemination, our project took roughly 18 months, compared to a planned timeline of 12 months. As the team’s confidence developed and ideas emerged, we spent time on unplanned activities. Although these changes improved the quality of our work, they also affected our projected timeline. We had to justify and explain this to key panel members and consider the wider impact of project adjustments. Jess had to balance the needs of a wide range of internal and external stakeholders. We feel that taking this time led to deeper relationships and a better end product than if we had tried to finish everything ‘on schedule’. Flexibility was a core principle of our project.
Dissemination
It was important to us that we shared and explained our findings to everyone involved in the evaluation. We gave a series of presentations to key panel stakeholders, during which we took questions. We then sent our report and executive summary to service customers who had taken part in our research.
The team also worked closely with the Support to Work service manager. We had meetings together to develop our initial recommendations into workable changes that were implemented within the service. We started this process early so that we could talk about agreed actions at the point of publishing our report.
Each of the team also had personal and professional contacts that they felt would be interested in our results. The geographical spread of team meant we could share the report with relevant networks across England.
We have also shared our experiences of the project’s co-production approach internally to Scope. We’ve written blog posts from our individual perspectives and have presented at a directorate meeting. We’ll also be contributing to further events about co-production at Scope as the organisation implements the approach more widely.
Co-Production at Scope
Prior to 2020, Scope operated on an Engagement and Participation model. Following the positive experience of our co-evaluation project and a small number of other co-production pilots, in March 2020 Scope’s Co-Production Lead and Strategy and Impact Manager made a proposal to our board and senior leadership to move to a Co-Production approach. The Board and Senior Leadership unanimously agreed and have committed to adopting co-production as strategic operational aim.
As part of the first year of this plan, our Strategy and Impact team are leading an organisational wide programme to raise awareness of co-production and facilitate its application through training, budgeting support, and targeted internal events and publications. We have also recruited staff from each directorate to be co-production champions, who will:
build understanding of co-production principles
facilitate discussions on adopting this way of working
support the setting up of co-produced projects in their teams.
We’d love to share ideas and experiences with any evaluation colleagues in ChEW who are interested in co-production. Please get in touch!
If you missed it, you can watch the session in the membership section of the ChEW website: Taking a Co-Production Approach to Evaluation(log in to access) .
Want to chat to other members about co production why not head to our membership forum here.