British Prime Minister Keir Starmer’s vow to prove his doubters wrong following heavy losses for his party in local elections have done little to ease calls for him to step down. 

“I’m not going to shy away from the fact that I’ve got some doubters, including in my own [Labour] Party,” Starmer said in a filmed speech Monday morning, as he took responsibility for last week's election results.

“I've spent too much time talking about what I am doing for working people and not enough time talking about why or who I stand for,” he added, promising to deliver the “change that we promised for a stronger and fairer Britain that we must build.” 

But several lawmakers are still adamant that someone else should be steering the ship as Labour recovers from its losses. Overall, more than 40 MPs [Members of Parliament] have urged Starmer to step down or establish a timetable for his departure.

“It was a passionate speech, but the content did not suggest anything close to the scale of change needed to rebuild communities like mine,” said Labour lawmaker Lorraine Beavers. 

“We need a democratic contest involving the most talented leaders from across our movement,” she continued, arguing that a cabinet appointment would “risk repeating the same mistakes already made.”

Although Starmer on Monday pointed towards what he views as symbols of success, including the fall in waiting times for the National Health Service (NHS) and his decision not to actively get involved with the Iran war, concerns over his premiership remain.

Starmer was already facing calls to resign over his handling of the appointment of Peter Mandelson, who was sacked months into his role as the U.K. ambassador to the U.S. Now, as Labour focuses on rebuilding following significant losses last week, many lawmakers within Starmer’s own party are calling for others to lead the way. 

Starmer threatened with leadership challenge 

Following the Prime Minister’s address, Labour lawmaker Catherine West said she was “collecting names of Labour MPs to call on the Prime Minister to set a timetable for the election of a new leader in September.” 

“The results last Thursday show that the PM has failed to inspire hope. What is best for the party and country now is for an orderly transition,” said West, describing Starmer’s Monday morning comments as being “too little too late.”

West’s response, although strong, was a step down from her previous threat to bring forth a leadership challenge.

Over the weekend, West—a former minister—had issued an ultimatum to Starmer's cabinet, saying if they didn't launch a bid to challenge his leadership, she would do it herself.

West and Beavers are far from the only members of Starmer’s own party who he failed to win over with his speech.

Lawmaker Nadia Whittome said the Prime Minister “should do the responsible thing for the country and our party, and outline a timetable for his departure with a democratic leadership contest.”

Paulette Hamilton, another Labour lawmaker, issued a similar statement Monday afternoon.

“The public have made it clear that they do not wish to hear further talk of a ‘reset’ from Sir Keir Starmer. Confidence is lost. Voters have stopped listening,” she said. “Now, an orderly transition must follow, and change must come from the top.”

Amid the calls for Starmer to go, some are focusing their attention on other potential options.

Andy Burnham, the Mayor of Greater Manchester, is receiving growing support—although he would need to be allowed to stand as a by-election candidate so he can return to the House of Commons before being a viable candidate.  

Mayor of London Sadiq Khan has backed the idea of Burnham returning to the Commons.

“If Andy still wants to return, and an opportunity arises he should be allowed to stand… I’m a firm believer in the team that I support winning, you want your best players on the pitch,” he said.

Angela Rayner, former deputy leader of the Labour Party, has also pointed to her support of Burnham, admitting that what the party is currently doing “isn’t working.”

"It’s time to acknowledge that blocking Andy Burnham was a mistake. We must show we understand the scale of change the moment calls for—that means bringing our best players into Parliament,” she argued.  

Burnham attempted to become a Member of Parliament earlier this year, but he was blocked from standing in a by-election by Labour's decision-making body, the National Executive Committee (NEC).

TIME has reached out to Downing Street and Burnham’s office for comment.

The process for challenging a Labour leader, explained

If someone were to trigger a leadership challenge, a candidate (who needs to be a current Member of Parliament) must receive support to run from at least 20% of sitting Labour MPs, which currently translates to 81 lawmakers, according to the Institute for Government.

“Once the runners and riders are decided, the contest goes to Labour's membership and its affiliated societies, and trade union members,” Tim Bale, a professor of politics at Queen Mary University of London, tells TIME.

While many see Burnham as a likely contender, Bale says that Rayner and Health Secretary Wes Streeting—who has reportedly shown interest in taking over the premiership—could be in the running.  

Lawmaker Al Carns is also a possible candidate, says Bale, who describes the Minister for the Armed Forces as a “first-time MP with an impressive military record.”

According to Bale, “Burnham is probably the favourite,” with those on the left of the party likely to prefer Rayner, and those on the right preferring Streeting. 

Cairns, Rayner, and Streeting are all sitting MPs, making their potential path to Downing Street already established. But Bale emphasizes that the only scenario for Burnham to join a potential contest would be if a current Labour MP were to stand down, “triggering a by-election in which Burnham can stand and [potentially] win as Labour's candidate.”

Outside of a leadership contest, the only other avenue for a change in Prime Minister would be a more aggressive ousting from top lawmakers. 

“Cabinet ministers could start resigning en masse, making it difficult—if not impossible—to replace them,” says Bale, referencing a similar scenario that occurred in 2022, when Prime Minister Boris Johnson, then-leader of the Conservative Party, stepped down after more than 50 members of his government resigned in protest against a series of ethics scandals.