Britain is not ungovernable, but the chalice of high office has been spiked with unusually fast-acting poison. Six prime ministers down in a decade. The spectacle of the lectern planted outside No 10 for a resignation speech has acquired the familiarity of ritual.

Since the Brexit referendum, the average tenure in Downing Street has been less than two years. That ballot isn’t directly responsible for ending Keir Starmer’s reign. He brought deficiencies to the job that have nothing to do with the EU. He took power without a clear sense of what he wanted it for and resented the expectation that he explain himself better. But those weaknesses were more cruelly exposed in our parched post-Brexit climate, a decade into the goodwill drought.

Governing is harder when resources are squandered on an exercise in self-harming statecraft. Disentangling the UK from the single market and building new systems to impede trade were processes that burned through reserves of diplomatic capital and economic credibility. The cost in foregone growth – the measure of how much richer Britain would have been on the pre-referendum trajectory – is estimated to be in the range of 4% to 8% of GDP.

That number doesn’t include the emotional toll: the coarsening of debate; radicalisation and polarisation; the toxification of political culture by a movement that sold immiseration as liberation and, when it all went wrong, blamed the losing side for refusing to indulge the delusions of the winners.

Britain is not the only democracy to experience the economic imbalances and social dislocation that incubate populism. Historians will situate Brexit in the wider context of a pan-European and transatlantic nationalist backlash against the complacent style of liberal globalisation that thought it had already won its final ideological battle when the Soviet Union collapsed.

The syndrome is not unique to any one country, but there are local idiosyncrasies. Its manifestation through a referendum is a complicating factor for Britain. There is clarity of intent when a radical party or demagogic candidate surges in an ordinary election and also a built-in recognition that the result is conditional on performance. Voters can change their minds next time. A rival electoral coalition might get its act together, as eventually happened in Hungary this year to oust Viktor Orbán after 16 years. A plebiscite contains no promise of next time. On the contrary, the supposed finality of the mandate was the leave side’s trump card. No volume of evidence to indicate that Brexit was a mistake could puncture the assertion that living with the decision forever, regardless of the consequences, equated to non-negotiable fulfilment of popular will.

The scale of those consequences was also obscured by the question on the ballot paper and barely elucidated during the campaign. None of the grievances that mobilised the leave vote were ever going to be satisfied by the loss of EU membership. Making the country poorer and surrendering a seat on the continental steering committees that enhance UK power in the world was not the way to “take back control”.

That misdirection has ugly consequences. The rhetoric of a national liberation struggle against an imagined colonial oppressor in Brussels turned Brexit into a competition to define patriotic purpose. When realisation of the winners’ stated goal led to a dead-end, they needed new enemies to sustain the cause. It is an old story. When revolutions fail to fulfil their promises, it must be the fault of saboteurs, traitors, “enemies of the people”.

The longer the disappointment, the more aggressive the hunt for scapegoats. Nigel Farage is not luxuriating in the long afterglow of his visionary achievement 10 years ago, but busily exploring ever more vindictive ways to cast immigration as the cause of the nation’s misfortunes. Racial animus that was implicit in the Brexit campaign – not subtle but coded with a certain deniability – is now unambiguous. Reform UK housing policy promises to reverse “anti-white” bias in the system, revoking the settled immigration status of hundreds of thousands of people, making many of them eligible for eviction and then deportation. It is a blueprint for freeing up social housing stock by means of ethnic cleansing.

Starmer’s inability to grapple with that radicalisation will be the sorriest part of his legacy. In opposition, he internalised the taboo on questioning the wisdom of Brexit, on the grounds that voters in former Labour electoral strongholds had been captured by the leave side in a culture war and could only be addressed in those terms. In government, that assumption fuelled creeping Faragism at the Home Office. The prime minister replicated the doomed Tory strategy of trying to win voters back from Reform UK by amplifying and validating the arguments of its leader.

It would be unfair to say Starmer did nothing to push back against the normalisation of far-right rhetoric. But, as so often, he found the right path by exploring the wrong one first. Only after he had provoked fury and despair in his own party, sounding like Enoch Powell when warning that Britain risked becoming an “island of strangers”, did he change the tone. His speech at last year’s Labour conference was packed with corrective intent. He spoke of a patriotism rooted in “love and pride [that is] about serving an interest that is more than yourself”. He contrasted this with the Farage model of relentless negativity, fomenting hatred because his ambition relies on the suppression of mutual understanding.

But Starmer’s voice didn’t carry beyond the auditorium, and he had no idea how to project it with sustained emotional resonance. On that score, Andy Burnham, assuming he is the successor, starts with the advantage of a more natural storytelling manner. Of course it will take more than narrative proficiency to drain the nationalist venom coursing through the body politic. Communications failure starts with ill-designed policy. Authenticity flows from surety of purpose. It is hard to sell something with sincere passion when you don’t really know what it’s for. Burnham could still get that wrong. But it helps that he knows who he is against. His parliamentary licence just to be a candidate for the top job, his mini-mandate, was earned in combat with Reform UK in Makerfield.

Starmer came to power thinking he could put Brexit behind him. He didn’t see it as a competition between modes of national identity. He didn’t see how the real sequel to the referendum struggle was not an argument about the UK’s relationship with the EU, but a battle to reclaim patriotism from the ideological movement that defines it with rage and racial segregation. It is a fight Burnham can win in a country where most people don’t want to deport their friends and neighbours. It is a fight he has to win if he wants to beat the Brexit curse and stay in Downing Street long enough to achieve anything else.

• Rafael Behr is a Guardian columnist