What story does Britain tell itself about Brexit, 10 years after the vote that transformed the country? Watch TV or read the papers and you find one of two viewpoints: from the common room or the conference room.

The common room story is about chums and how they fall out. Friendships forged on hallowed playing fields and over Cotswold kitchen suppers, then dashed on the rocks of ambition. The new BBC documentary Brexit: A Very British Civil War is the latest in the genre, recounting what Dave said to Boris said to Michael said to Dom. It oohs at the deals struck over sets of tennis, and aahs at the then prime minister threatening dissenters with: “I will fuck you up for ever.” This is David Cameron as box office: the Scarface of the Bullingdon Club. And Brexit, you understand, was simply an Oxford fracas that got out of hand.

Over in the conference room, the suits bemoan the damage to GDP and whether one day the European club might let us back in. Brexit, you see, is about trade and contracts.

Entirely missing from either story is the streets. Those rainy streets outside polling stations where on 23 June 2016 people queued up, many of them for the first time in years. The slim majority opting to leave did so not necessarily because they cared much about the EU, but as part of a giant vote against the establishment. For months the government, the opposition, the Treasury’s Project Fear department, the Bank of England, the Confederation of British Industry, the Trades Union Congress and individual employers said Britain would be Stronger In – and still the public turned out to give their masters a huge raspberry.

In the weeks and months afterwards, politicians, newspapers and thinktanks constantly tried to show they were in tune with this public anger. “We hear you!” said one politician, a guy called Andy Burnham who had recently lost his second bid to become Labour leader. In response, Boris Johnson later dabbled in something called “levelling up”, which turned out to be a slogan rather than a policy. Ten years later, this revolt has been struck from the official narrative.

Yet it is this Brexit story, the story of the streets, that is easily the most important of the three. It directly shapes our politics today. You can draw a clear line from the mutinous mood of that summer to today’s byelection in Makerfield, Wigan, where in 2016 almost two out of three voters went leave. Ten years ago, voters were learning to voice their distrust of the two-party system. Politicians of both colours had lied to them about the stability of their banks, about austerity ending soon and wages rebounding – and now they wanted revenge. This week in Makerfield, not one candidate on the ballot paper supports the prime minister. Reform UK and Restore Britain, two rightwing parties that didn’t even exist a few years ago, will take a lot of votes.

I remember seeing that anger up close. Some weeks before the actual referendum, when the silverbacks of Westminster were still sure they had it in the bag, I reported around south Wales. What I saw and heard convinced me that this Labour heartland would vote against their party’s advice and for the cause fronted by Nigel Farage, in an act of self-harming nihilism. I remember visiting the village of Llanhilleth, and visiting the Miners Institute that towered like a cathedral over the workers’ cottages. It had been built with miners’ subs and then, in recent years, rescued from dereliction with Brussels money.

Inside was a barrel-chested man called Gareth Meek, who pulsed with anger – whether just towards this Asian lad from a posh newspaper or others, I don’t know. He used to be in carpets, until his back gave out; now he was here in this empty monument of better times. He was voting leave, he said, although his anger wasn’t aimed at the “Eurocrats” but Margaret Thatcher and Tony Blair and Cameron. “They sold the country out.” How would leaving help? He shrugged. “The damage is already done. You ain’t going to pull that back now.”

Not everyone who voted for Brexit was from a dead factory town. I sometimes characterise that vote as two groups – the postindustrial and the retired, both now disconnected from the formal economy. At a fete in east Dorset, a well-to-do pensioner told me all about the migrants overrunning London, before admitting she barely ever visited the place. “I just want my country back,” she said. Again, that twitch of rage.

The politics expressed itself as anti-establishment and anti-immigrant, although it often seemed to me even then that Romanians were merely the proximate target: it was mainly about feeling you and your town had fallen too far down the pecking order. While running a Vote Leave street stall, Callum Vaga told me: “I look out for men, especially from ethnic minorities and most of all if they appear less well-off.” But even in 2016, anti-immigrant feeling could easily turn into violence. The murder of Jo Cox by a neo-Nazi was the most horrific example but a few days after the vote, Polish-origin schoolkids in Huntingdon got cards calling them vermin.

Ten years ago, it was “breaking point” posters; today, it is “pure cold rage”. The extremism was always there, but Westminster and the information economy has allowed it to become mainstream. In a forthcoming survey for Hope Not Hate, people were asked if “violence can be necessary to defend something you strongly believe in”: 29% agreed.

Rather than tackle the economic and social basis of that disaffection, Westminster left it to Farage to turn it into ethnic resentment. Even today, Starmer’s EU reset is solely aimed at the conference room. For the streets, he took his cue at the outset of his leadership from the branding agency that advised him: “The use of the flag, veterans … give voters a sense of authentic values alignment.” What began 10 years ago as an anti-elite revolt lives on today with a political establishment chasing their voters for fear of being left behind. As the French revolutionist Alexandre Auguste Ledru-Rollin is supposed to have said in the tumult of 1848: “There go the people. I must follow them, for I am their leader.”

• Aditya Chakrabortty is a Guardian columnist