Evaluation type
What does this project involve?
Your Choice (YC) provides training for youth practitioners in Cognitive Behavioural Therapy (CBT) techniques, who then aim to provide 3 sessions per-week, for 12 -18 weeks to young people. Developed by the London Innovation and Improvement Alliance (LIIA), and supported by the London Violence Reduction Unit, youth practitioners receive 5 days of training and an accompanying handbook and resources, followed by monthly supervision from a Clinical Lead. Practitioners’ meetings with young people aim to build trusting and authentic relationships, using CBT techniques and tools to support the young person to better understand themselves, equip them with coping strategies, and empower them to disrupt unwanted patterns of behaviour. In this project, 11-17 year olds assessed by a multi-agency risk panel as having a medium or high risk of harm or vulnerability as a result of extra-familial harm were referred to the programme.
Why did YEF fund this project?
Cognitive Behavioural Therapy (CBT) is a type of talking therapy that has been used to address a range of psychological difficulties. When it is applied to violence prevention, it is based on the idea that negative or impulsive thoughts and behaviours might make someone more likely to lash out or act aggressively. As the YEF Toolkit explains, CBT is associated, on average, with a ‘High’ estimated impact on violent crime.
However, we lack robust evidence of impact in an English and Welsh context. In addition, we recognise the current capacity challenges in the mental health system that may prevent enough clinicians being available to directly deliver CBT to at risk young people, while we know that the highest risk young people are currently least likely to access such support in a clinical setting.
Your Choice offers an alternative approach, that recognises these challenges, while aiming to provide those in need with CBT-informed support. Specifically, youth workers receive training and supervision to deliver CBT-informed sessions in a non-clinical setting to young people at high and medium risk.
YEF and the Home Office funded a pilot study of YC. The evaluation aimed to assess the feasibility of effective data collection, examine the implementation of YC, assess the adherence of Local Authorities (LAs) and youth workers to randomisation, and pilot study outcomes and evaluation methods. The evaluation used a survey of young people and practitioners before and after the programme that featured a range of measures (including the Strengths and Difficulties Questionnaire (SDQ), the Social Connectedness Scale (SCS-R), and the Short Warwick–Edinburgh Mental Well-being Scale (SWEMWBS). Session forms were also monitored, and interviews conducted with practitioners and participants.
Each LA was asked to name two teams of youth workers, who work with the children targeted by YC, who could be available to be trained. These two teams were then randomised to be treatment or control teams, with treatment teams receiving YC training. Once young people were identified by a multi-agency panel as requiring support, the intention was for them to be referred to the team with the most capacity at that point. 13 London LAs participated in the pilot trial and randomised teams into treatment and control teams. Overall, 27/32 London LAs named two or more teams to participate in YC, but only 13 LAs (comprising 300 young people and 65 practitioners) participated in the pilot. 34% of the 300 young people identified as Black, 30% as White, 10% as Asian, 1% as Arab and 2% as Other. Pilot recruitment began in July 2022, and at the time of writing the pilot was ongoing. This report represents an interim findings report.
Key conclusions
| Data collection in the pilot study has proven very challenging. Although all LAs agreed to support and contribute to survey data collection, the collection of data has been slower than anticipated. This was a particular problem among control teams. Recruitment of young people was also lower and slower than anticipated. The evaluator suggests a range of measures to improve data collection in future trials. |
| Overall, YC was implemented in accordance with the logic model and intervention description. The training was successfully delivered to the treatment teams, and most young people selected to participate were at high or medium risk. |
| Implementation was facilitated by practitioners’ understanding of and commitment to YC, the funding provided for delivery, the flexibility of the programme that enabled the use of creative and engaging sessions and the monthly supervision provided by a clinical lead. Implementation concerns included the lower-than-anticipated frequency of meetings and the additional workload associated with data collection. |
| The extent to which practitioners followed the protocol for assigning young people to teams was mixed. The design – established to ensure an ‘as good as random’ approach, where participants were assigned to the team with the most capacity, irrespective of their need – seems to have been broadly followed by the LAs who participated in the pilot; however, there was some evidence that some children were allocated to YC based on their perceived greater need – which would introduce systematic bias. Twenty-three per cent of control sessions also delivered YC techniques. |
| The measures used in the survey were acceptable. Greater concerns were raised regarding the evaluation methods. Of the LAs who delivered YC, 10 had a high, six had an intermediate and 15 had a low readiness for trial. |
What will YEF do next?
YEF is proceeding to fund an efficacy trial of YC. This trial will attempt to implement the range of recommendations suggested by the evaluator to improve the quality of evaluation.