School of Physics and Astronomy

Evaluation

Context

Research has shown that one-off outreach interventions have little to no impact on the students involved. Therefore, recently there has been increased effort in devising outreach activities which incorporate repeat interventions to combat this effect. However, so far most of these series of interventions have still shown no statistically significant changes in students' attitudes or aspirations. While students often enjoy science and think its important, one major barrier has been that they do not see it as part of their identity. Programmes which offer the opportunity for students to identify more with science within their peer group are thought to be one way of addressing this barrier. Furthermore, STEM subjects such as Physics continue to face gender balance issues with it historically being seen as a 'male subject'.

The Wellcome Trust's commissioned survey of independent research projects found that they have benefits to both students and teachers, but still are only undertaken by a small minority of students nationwide and globally with such projects mentored by universities or industry only sometimes. The Institute of Physics gender balance programme has found that participating in research projects has a positive effect on the uptake of physics by girls.

The School of Physics and Astronomy's PRiSE programme, by design, thus has the potential to tackle many issues facing STEM outreach. We undertake comprehensive evaluation of the programme each year to evidence the positive impact we have upon all involved. Here we detail some of the findings of this work.

Schools

One criticism of independent research projects has been that they are only accessible to the most privileged of students. Sometimes projects require expensive equipment which only the most well resourced schools can afford to purchase/hire. Furthermore, with a lack of expert support and mentoring in the majority of cases it stands to reason that the vast majority of students would be unable to effectively take part.

Our programme aims to be accessible to students of all backgrounds: we work with a diverse variety of different schools, support the projects throughout, and do not charge schools to take part. The schools we work with has steadily expanded to span much of London and the surrounding region and various metrics show that we are indeed addressing the widening participation agenda in STEM.

Distribution of free school meals percentages for schools we worked with compared to all schools nationally and other schemes

Distribution of indices of multiple deprivation scores of schools' local (dashed) and catchment (solid) areas

We offer a near unprecedented level of support for these projects including kick-off workshops, comprehensive introductory guides tailored for either students or teachers, as well as various additional online resources all of which are freely accessible to all. One of the key differences with PRiSE compared to other providers of independent research projects is the involvement and support from active researchers, an element which has been deemed by both students and teachers as the most important way of facilitating these projects to all. However, we still stress the importance that students and teachers continue to independently work on, learn and problem-solve their projects outside of contact with researchers. This is another element that sets PRiSE apart from schemes which focus more on simply following prescribed tasks, often for crowdsourcing / citizen science purposes.

Impact on students

One of the aims of the PRiSE programme is to develop students' awareness of current areas of Physics research and the related research methods involved. After having had first-hand experience of both of these for an extended period of time through working on projects with us, we have found that students report having increased their confidence in various topics related to their project area and the scientific methods which apply. These increases in confidence are highly statistically significant increases, with the overall results so far shown to the right along with the range of results for individual topics and methods applied to each project.

It was a very rewarding experience which allowed us an insight into the research conducted at University level. This helped us to develop crucial skills needed in the next years of our studies. It was truly amazing to hear how significant the event we found was and that it will be forming the basis of a proper scientific paper.
— Student from Eltham Hill School

We've also found that by taking part in projects, students are developing a wide array of different skills - many of which they would likely not have had access to previously in their school environment and may only otherwise be experienced at a university-level. Therefore, PRiSE is enabling the future workforce to upskill themselves through support by their teachers and engaged academics. Finally, students generally report having had a positive and rewarding experience through taking part in our programme.

Word cloud of students' and teachers' experiences of taking part in research projects (logarithmic scale).

Word cloud of skills developed by students through their research projects (logarithmic scale).

We have started investigating the long-term impacts taking part in the programme has had on students' aspirations and trajectories. These have shown some promising results so far and we hope to present more robust outcomes in the future.

I am now pursuing a Physics degree from Cambridge. Thanks for helping me find my enthusiasm for Physics.
— Student from Newham Sixth Form College

Impact on teachers

As well as being a great experience for students, we've found that teachers also benefit through their role in integrating the projects into their school and providing their students with encouragement and support. Teachers have fed back that the level of support provided by the researchers, uncommon in many other independent research projects, has enabled them and their students to successfully take part. Teachers have reported learning new physics content and developing skills which can be implemented or referred to in future lessons. Some teachers have also gained confidence in supporting extra curricular activities and discussing unfamiliar areas in physics or research content with their students by taking part. Most teachers continue running projects with us for multiple years at their school, thereby developing a lasting relationships between their school and an HE body.

I am now more aware of what our students are capable of - not just listening to visiting speakers but being actively engaged in real-world research!
— Teacher

More information

Further details about all of the above and our evaluation can be found in the following places:

PRiSE Pilot Evaluation Report 2016 [PDF 1,145KB]
Archer, M. O. (2017) So you're looking to run a research in schools project? Practical tips from the evaluation of a pilot programme, SEPnet Report, doi:10.13140/RG.2.2.25674.06088

Paper on students' discovery of space sounds following solar storms
Archer et al. (2018) First results from sonification and exploratory citizen science of magnetospheric ULF waves: Long-lasting decreasing-frequency poloidal field line resonances following geomagnetic storms, Space Weather, doi:10.1029/2018SW001988.

Other articles:
Royal Society Schools Network Blog Post
News story about 'Cosmic Con 2018' from one of the students
BBC Radio 4 Inside Science featuring MUSICS project and students' discovery

Learning from the PRiSE programme has been presented at the following:

American Geophysical Union Fall Meeting 2018
European Geosciences Union General Assembly 2017, 2018, 2019
European Week of Astronomy and Space Science 2018
Geospace Environment Modeling Workshop 2018 (x2)
Interact symposium 2017, 2019
International Space Science Institute, Bern Team Meeting 2016
Magnetosphere Ionosphere Solar-Terrestrial Autumn Meeting 2016, 2018
NASA THEMIS Mission Science Working Team Meeting 2018
Royal Astronomical Society 2019
SEPnet Outreach and Public Engagement Meetings 2015, 2016, 2017, 2018
SEPnet Outreach and Public Engagement 10th Anniversary Symposium (2018)
University of Surrey 2018
The Ogden Trust Outreach Meetings 2016, 2017, 2018