Teaching pupils in classes grouped by ability improves the results of high-flyers but does not affect the progress of less able children, according to a study that upends decades of debate over mixed-ability education.

The research by University College London’s Institute of Education found that secondary school pupils in England with previously strong maths performances made slower progress in mixed-attainment classes than when they were taught alongside children with similarly high ability.

Crucially, the study backed by the Education Endowment Foundation (EEF) showed that setting by ability did not “significantly harm the attainment of low-prior-attaining or socioeconomically disadvantaged” pupils.

The study’s impact analysis showed negative effects on self-confidence in maths for pupils in mixed-attainment schools, compared with those in schools using setting – challenging previous reports that setting harms the confidence of those outside the top sets.

John Jerrim, professor of education and social statistics at UCL, who has studied the effects of mixed-ability classes but was not involved in the new research, described the outcome as “big and important”.

“The EEF have spent huge amounts of money investigating this issue. I think they should probably now come out and support achievement grouping in maths,” Jerrim said, arguing that “it has no negative impact on lower-achievers, some positive benefits for high-achievers, and helps teachers manage workload”.

Jerrim added: “It wasn’t long ago that some educational researchers in the UK and Ireland were calling ability grouping ‘symbolic violence’. I think this work – and other evidence – shows the need for more restraint from academics.”

Becky Francis, the EEF’s chief executive, said the research was the first to detail the relative progress made by pupils at different levels of ability for those in mixed-ability classes and those in sets.

Francis said: “Basically, this direct comparison between setted classes and mixed-attainment is what is innovative and revealing about this new study.

“What we found is that there was very little difference for low-attaining young people between mixed-attainment and setting classes.

“Meanwhile, for the high-attainers who make strong progress in high sets, they make lower progress in mixed-attainment classes overall.”

The study looked at mathematics attainment and self-confidence for year 7 and 8 pupils, aged 11 to 13, attending state schools in England. The research was based on results from 28 schools with mixed-ability classes and 69 similar schools that used setting by attainment.

Among high-achieving students, those in mixed-ability classes made two months’ less progress on average, compared with students in schools using setting by attainment. Schools with mixed classes made one month’s less progress overall.

The researchers noted: “In general, despite well-intentioned policies in mixed-attainment schools around equity and challenge for high-attainers, only setting schools appear to be challenging high-prior-attaining students.”

While mixed-ability schools had a smaller gap in results between their best- and worst-performing pupils, the researchers said it appeared to be “driven by lower progress among the high-prior-attaining students, rather than greater progress by the low-prior-attaining group”.

The study also warned that for ability setting to work properly, schools must avoid allocating their best teachers to the top sets.

Pepe Di’Iasio, the general secretary of the Association of School and College Leaders, said: “School leaders are best placed to make decisions about setting, as they best know their context and needs of their pupils. We are sure that this research will be extremely helpful in informing those decisions.

“The essential ingredient is, of course, having sufficient numbers of specialist maths teachers to ensure that pupils at all attainment levels receive the best support possible.

“Unfortunately, there is a longstanding problem with recruiting maths teachers and many schools have no choice other than to use nonspecialists and supply cover.”