Evaluation type
What does this project involve?
Level Up clinicians delivered four group-based, online sessions to children and their parents/carers during the summer holidays between Year 6 and 7. The sessions aimed to prepare children for the transition to secondary school by helping them manage their behaviour, emotions and relationships. They were also accompanied by online activities, optional art therapy classes and a final session at the start of Year 7.
Why did YEF fund this project?
The transition from Year 6 to Year 7 often causes worry and challenge for families and may represent a critical period for children. Teachers have also identified the importance of a smooth transition in ensuring that children remain safe and avoid risky and violent behaviour. Problems with successfully transitioning to secondary school can lead to lower education outcomes, school drop-out, increased depression and anxiety, and increased involvement in violent and anti-social behaviour.
The Tavistock and Portman NHS Foundation Trust developed a new intervention, Level Up, to help children over this transition. Given its potential to reduce youth violence in the long-term and the lack of robustly evaluated programmes (of a similar nature), the YEF funded an evaluation of this new intervention.
The evaluation of Level Up was a feasibility and pilot study. The feasibility study aimed to ascertain what Level Up’s core activities are and what barriers and facilitators to delivery exist. It also explored perceptions of the programme. These questions were answered using semi-structured interviews with families, school staff and Level Up clinicians. 35 families participated from May to December 2020. Children participating in Level Up were identified by school staff as having elevated levels of emotional or behavioural difficulties, but not meeting the threshold of need for child and adolescent mental health services (CAMHS).
The pilot study then aimed to explore how Level Up delivery and evaluation could be scaled up further, attempting to identify how a future trial could be designed, while also exploring whether there were early indications that Level Up may be associated with positive outcomes. Questionnaires, attendance data, and semi-structured interviews were used to address these aims. 72 families participated in Level Up during the pilot study, which was delivered from May to December 2021.
Both the feasibility and pilot studies were undertaken during the COVID-19 pandemic, requiring both the delivery and evaluation teams to adapt to challenging circumstances.
Key conclusions
| Level Up clinicians, school staff and families reported several barriers and facilitators to the programme during the feasibility study. Regarding recruitment, remote working and challenges identifying the target population posed barriers. Regarding delivery, facilitators included building good relationships between the Level Up team and families, as well as between families themselves, and the online format increasing programme reach. Barriers included moving the intervention online, communication barriers with families and varying levels of attendance. |
| Families in the feasibility study identified a range of perceived benefits, including enjoying sessions, appreciating a space for sharing and learning from others, perceptions of a smooth transition to secondary school and filling the gap left by the coronavirus lockdown. |
| The pilot study identified that changes would be required to measure the impact of Level Up, including clarifying mechanisms of change and outcomes and the inclusion of measures to capture school readiness or connectedness. Assuming a small effect, a 620-family sample would be required in a randomised controlled trial. |
| Level Up was perceived as enjoyable and helpful by families in the pilot study, and families perceived improvements in confidence and reduced worries over the transition to secondary school. Some minor negative consequences were mentioned by a minority of families, including uncertainty about the inclusion criteria for Level Up or feeling bored during sessions. Families’ attendance at Level Up varied during the pilot study. On average, children attended about half of the sessions (2.4/5) and parents/carers even less (1.9/5). |
| The pilot study struggled to recruit and retain families. This could be improved by including in-person research visits to participants, questionnaire incentives and greater clarity for schools on selecting children to take part. Embedding the delivery of Level Up into school term time and timetables may also improve retention. |
What will YEF do next?
Level Up was well-received by those families interviewed, the majority of whom found it enjoyable and helpful. However, given the limitations discussed (such as attendance, recruitment and retention), a large-scale intervention and evaluation is not currently feasible. The YEF has no current plans to fund further evaluations of Level Up.