Scottish schools that are currently in receipt of funding from the Scottish Attainment Challenge or Pupil Equity Funding are in the position of trying to source interventions ‘…targeted at closing the poverty related attainment gap’. The National Operational Guidance 2018 provides information for schools on how to source appropriate interventions and in the ‘Scottish Attainment Challenge: Learning & Teaching Toolkit’ provides specific advice on the effectiveness and cost of 34 different interventions. However as schools seek to engage with different new projects and providers how reliable is this research informed advice?
Our research has involved the development and implementation of an intergenerational mentoring programme in Scotland seeking to assist young people from poor or working class backgrounds into higher education. The project has shown considerable impact, particularly in terms of enabling young people to enter the most competitive professions. As the project has evolved we are now in a position to introduce the project to new schools across Scotland.
A school looking to assess intergenerational mentoring as a potential intervention for children or young people will, via the National Operational Guidelines be directed to the Education Endowment Foundation (EEF) toolkit. Within the EEF toolkit ‘mentoring’ is rated 0 as ‘Very low or no impact for moderate cost, based on extensive evidence’. To qualify this the toolkit expands:
The impact of mentoring is variable, but on average it has tended to be low in terms of direct effect on academic outcomes. There is some evidence that pupils from disadvantaged backgrounds are likely to benefit more (nearly double the impact). Other positive benefits have been reported in terms of attitudes to school, attendance and behaviour.
However, there are also risks associated with unsuccessful mentor pairings which may have a detrimental effect on the mentee, and the negative overall impacts seen in some studies should prompt caution
On further reading the toolkit reveals that it ‘…collates research from both sides of the Atlantic‘ and the reader is informed that this is based on the evidence generated by 5 meta-analyses conducted in the past 10 years. In reference to work in the UK only one study is mentioned that worked with eight-to-nine-year-olds in Northern Ireland and ‘…found small improvements of about two months’ progress in fluency, but not in reading comprehension’.
How then can we apply this advice to a school interested in looking at the potential of intergenerational mentoring? The greatest danger is that the overall message from the toolkit, presented in tabular form with stars for impact, will label mentoring as not worth the effort and dissuade schools from engaging. This denies the diversity and complexity of mentoring and disregards the potential for impact of projects working with disadvantaged children and young people.
In terms of the evidence base drawn on the meta-analysis referred to have all been conducted in the US and will, as such, include many diverse forms of mentoring. In fact a clear message from this body of work is the difficulty in generalising from such diverse projects and sources of evidence. The meta-analyses don’t dissuade from engaging in mentoring but rather emphasise the need for mentoring projects to be careful in how they recruit mentors, train and support them and match their mentees and mentors. These factors are what is seen as crucial to achieving better outcomes in mentoring.
A more clear message from the toolkit would draw on the meta analysis to indicate that mentoring interventions engage with very different groups of children and young people with different levels of impact and, crucially for ‘vulnerable’ children and young people, different levels of risk. It would also indicate that high quality mentoring has potential for influencing positive outcomes for children and young people. It would advise schools seeking a high quality intervention to look at the ways in which projects are funded, how they are delivered and managed and what support they will offer to those engaging.
Unfortunately in this instance the EEF Toolkit over simplifies the data, it generalizes about ‘mentoring’ in a way that is unhelpful, likely to dissuade schools from becoming interested and ultimately prevent children and young people from having better support to realise their potential.