Preparing the next generation of evaluators for an AI-assisted world

Fiona Coventry, Ben Durbin, Vicky Fisher | Raspberry Pi

As AI becomes a part of our workplaces and evaluation design, we need to consider how to prepare young people for future jobs in evaluation. Twenty years from now, what knowledge and skills will evaluators need? To ensure evaluation is democratic, and to counter the risks of 'evaluation by tech', the next generation of evaluators will need to understand the societal and ethical issues involved with AI technologies, and how to critically evaluate them.

So how can we make sure that young people are developing these skills? 

About the Raspberry Pi Foundation

The Raspberry Pi Foundation is an independent charity with a mission to enable all young people to realise their full potential through the power of computing and AI. 

We support young people to:

• Develop the knowledge, skills, and confidence to use computers and advanced technologies like AI in their work, community, and personal life; to solve problems; and to express themselves creatively

• gain the ability to critically evaluate digital technologies and their application, and to design and use technology for good

• develop the mindsets to confidently engage with technological change and make decisions about the role they want technology to play in their lives

  
  

We do this through creating learning experiences for young people, providing resources and support for educators, and undertaking research and thought leadership. As part of our mission we have been integrating AI education content across our products to help increase AI literacy.

Our AI education initiatives

> Experience AI

Our largest initiative in this space is our AI literacy programme, Experience AI, which was developed in collaboration with Google DeepMind. Experience AI is a free educational programme that helps educators and students learn about artificial intelligence (AI). It includes lessons, classroom resources, and hands-on activities and aims to help students develop:

• A foundational understanding of AI technologies

• An understanding of the social and ethical implications of AI technologies

• An understanding of the role that AI can play in their lives

Experience AI is a truly global programme. The resources have been downloaded over 900k times across more than 190 countries or territories, and we’ve worked with a network of partners to localise and translate resources into over 20 languages. We’ve also worked with partners to train educators using a cascade model. We have trained partners in over 40 countries, who have gone on to deliver training to more than 45,000 educators. Those educators can reach an estimated 4.5 million young people. However, as evaluators you know reach is only part of the story.

We evaluate Experience AI using a mixed-methods approach comprising: 

• Resource download data from the Experience AI website, 

• Monitoring data from partner organisations

• Survey responses from partners, educators, and students 

• Qualitative data captured by partners through lesson observations, teacher interviews, and student focus groups

We’re also currently piloting approaches to collecting student assessment data. 

Some key headlines from our survey responses include:

• 91% of educators agree Experience AI increased their students’ knowledge of AI concepts

• 93% of educators agree the Experience AI training increased their knowledge of AI concepts

• 89% of students agree they better understand what AI and machine learning are

• 87% of students agree they know more about the benefits and risks of AI

These are backed-up by findings from qualitative feedback and lesson observations. The short video below gives a sense of how this plays out in real classrooms across the globe.

https://youtu.be/udnhu43tTfk

> AI education in non-formal learning

In non-formal learning spaces (learning outside the classroom e.g. after school clubs, organisations like the Scouts and competitions), we are also supporting young people to understand the principles behind machine learning and AI.

Our largest non-formal programme is Code Club. We support over 9,000 clubs worldwide reaching an estimated 320,000 young people each month. We provide all the support materials for clubs, such free project resources, including an AI and data pathway. Based on a small-scale study (134 clubs), 40% of our clubs are using AI projects with young people. 

Code Clubs and young people around the world can also take part in our global technology showcase Coolest Projects, which includes an AI category where young people combine their coding with AI systems. Example entries to the showcase this year include a team whose project was a classification tool for identifying and counting wetland birds, and a young person whose project was a tool to improve recycling by suggesting other uses for standard household rubbish.  

While we have not yet directly measured the impact of our AI education content within our non-formal learning initiatives, we believe that non-formal programmes help young people develop the skills they need to thrive in the AI-assisted future. 

• Developing skills: 97% of our mentors say that young people in Code Clubs develop skills and confidence in digital making. This was supported by an independent study in 2024, which showed young people increased not only their computing skills but also wider skills such as communication and problem solving.

• De-mystifying technology: As one mentor put it, young people learn to “build rather than just consume”.

 "It helps them understand what's behind the technology rather than just using it" 

– Code Club mentor in Italy

Challenges and barriers

AI education is in its very early stages and therefore not without challenges. Key challenges we have observed from our evaluation findings include:

• Educator confidence: Educators frequently lack confidence to introduce young people to AI.

• Curriculum lag: AI is a very fast-moving area, resulting in national curricula lagging behind in many places. This means educators can struggle to fit AI lessons into already busy timetables. 

• Access to infrastructure: Where schools or clubs lack access to up-to-date equipment, internet, or electricity, young people miss out on opportunities for hands-on practice with AI tools and are at risk of falling further behind their peers with better access. 

Implications for education and the evaluation community

The next generation of evaluators will need the technical skills to use AI in their work, but will also need to be able to make decisions about how to use it well and the potential risks involved.

If the future evaluation community is to be representative, inclusive, and rigorous, we must ensure that AI literacy is accessible to everyone, not just a privileged few.

To achieve this, we need to champion three key areas:

• Guidance and resources for educators, including non-specialists, so that they are confident to bring this knowledge to young people

• Opportunities for young people to learn about AI through policy environments that enable integration of AI into formal curricula, supplemented by non-formal opportunities

• Infrastructure equity, to remove barriers to access