London - British Prime Minister Sir Keir Starmer insisted Tuesday that he will "get on with governing," rejecting growing calls from within his own party for him to step aside, which have been intensified by multiple resignations from his cabinet.

In the wake of disastrous results in local elections last week, a fifth of all members of the U.K. Parliament from Starmer's own Labour Party have now called on him to step down, and four members of his cabinet resigned Tuesday in an effort to force him to quit.

Jess Phillips, the national minister for safeguarding; Alex Davies-Jones, minister for victims and tackling violence against women and girls; Zubir Ahmed, Parliamentary Under-Secretary of State for Health Innovation and Safety; and Miatta Fahnbulleh, minister for devolution, faith and communities, all announced their resignations on Tuesday.

"People are crying out" for change, Fahnbulleh wrote in her resignation letter to the prime minister. "The public does not believe that you can lead this change – and nor do I."

CBS News' partner network BBC reported Tuesday that more ministerial resignations are expected in the coming days.

On Monday, six lower-ranking ministerial aides quit, and several senior members of Starmer's governing cabinet urged him to set out a plan for his resignation and to hold a party leadership contest, according to The Guardian newspaper.

But speaking Tuesday morning with his cabinet ministers, Starmer was defiant. 

"I take responsibility for these election results and I take responsibility for delivering the change we promised," said the embattled prime minister. 

"The country expects us to get on with governing," Starmer added. "That is what I am doing and what we must do as a cabinet."

If 20% of all the Labour Party's elected members of parliament can agree on an alternative figure to head the party — which, with Labour's current 403 seats in the House of Commons, would mean 81 MPs — they can effectively force him out of the role, and the country's top job with it.

At least 80 MPs have publicly called for Starmer's resignation, according to the BBC, but they have not all backed the same alternative candidate. And it may be a very high bar to clear, with at least three names of potential Starmer replacements being bandied about by politicians and political pundits.

Starmer confirmed Tuesday that the Labour Party's process for challenging a leader had not been triggered by unified dissent.

If that were to happen, he would be able to stand against any challengers in a leadership contest, in a process similar to U.S. primaries. Alternatively, if a majority of Starmer's cabinet decide he's not the right man to lead the party forward, they could resign en masse and make his position untenable. 

How did it come to this?

In a May 7 round of elections for local council seats in England and seats in the semi-autonomous legislatures of Wales and Scotland, Starmer's Labour Party lost big. It not only hemorrhaged 1,000 council seats in England, but also lost its firm grip on the Welsh legislature, which it had controlled for 27 years.

The big winner on the day was populist, anti-immigration party Reform UK, led by President Trump's ideological ally Nigel Farage. Reform won nearly 1,300 local seats in England, made huge gains in Scotland and came second in Wales. 

The elections were seen as a referendum on the current government, not unlike the U.S. midterms, and they were an unmitigated disaster for Labour — and personally for Starmer, a former government lawyer whose popularity has spiraled downward since he took power in 2024. 

Starmer tried to claw back the initiative in a speech on Monday. With his shirt sleeves rolled up, he vowed to rebuild Britain's relationship with Europe and nationalize the flailing U.K. steel industry — and he warned the U.K. could "go down a very dark path" if Labour failed to recover from its election losses.

"I know I have my doubters and I know I need to prove them wrong," he said. "And I will do so."

But some lawmakers from his own party seemed unimpressed by the effort, with various parliamentarians telling the BBC that the speech, "made me feel sorry for the PM," that it "really didn't cut the mustard," and, perhaps most damningly: "Meh."

Another MP said it came across like Starmer "delivering a planning application" — a familiar critique of a politician who has always come across as more managerial than charismatic. 

The shadow of the Epstein scandal

Starmer's government, despite a massive Labour majority in parliament, has struggled to lower the cost of living and kick start a floundering British economy — hampered albeit by the wars in Ukraine and Iran and their devastating impact on global energy prices.

But the pointed criticism of him as an individual leader really started gaining steam in April, when Starmer was widely criticized over his decision to appoint a former friend of Jeffrey Epstein as the U.K.'s ambassador in Washington. 

Opposition politicians insisted that Starmer was either lying about what he knew, or guilty of ineptitude for not knowing sooner of Peter Mandelson's close ties with the late American financier and convicted sex offender. Several opposition lawmakers started calling for him to resign then.

Issues "deeper than the prime minister"

Even now, there are Labour politicians who feel that sacking Starmer may do little to address the underlying problems the party is facing.

According to the BBC, more than 100 Labour MPs had signed a letter by Tuesday backing Starmer to remain in his position.

Stephen Houghton, who last week was booted from his position as Labour leader of the Barnsley city council in the north of England — one of hundreds of local officials who lost posts to members of the surging, populist, anti-immigration Reform UK party — said the problem "goes deeper than the prime minister."

Speaking of the Labour rout in the local elections, Houghton, who lost the council leadership but managed to cling onto his own seat, said the blow had "been coming for 30 years around the country, in post-industrial communities, coastal communities, that have been left behind."

"You can change prime ministers all day long," he argued. "If you don't change policy, it's not going to charge."

"This is not just about this government, previous governments have ignored the problem, too, but I will be saying to Keir Starmer that he needs to deal with this now, because if he doesn't, there will be even more losses like this coming soon."