The announcement of Peter Mandelson’s appointment as US ambassador ensures that 20 December 2024 will be recorded as a fateful day in Sir Keir Starmer’s leadership. Less remarked on, but relevant in hindsight, is a speech that the prime minister made earlier that month to launch a “plan for change”. Sir Keir set out ambitions to improve public services and lamented caution in the civil service. Whitehall, he said, was too often comfortable “in the tepid bath of managed decline”.

The prime minister was feeling thwarted by the machinery of government. In that context, it is easy to see how he might have been persuaded that Lord Mandelson would make a better emissary to the US than the long-serving professional diplomat in post at the time. Impatience with a slow-moving apparatus is conveyed also in the account given by Sir Olly Robbins, the former permanent secretary at the Foreign Office, of a department under “constant pressure” to complete Lord Mandelson’s security vetting. The prime minister told the Commons on Wednesday that no such pressure existed.

Sir Keir now wishes that the process had not been expedited so efficiently. He sacked Sir Olly for granting the necessary clearance despite the vetting process raising red flags, although doing so appears to have been a fulfilment of Downing Street’s unambiguous will.

This is a common syndrome. Ministers complain that the bureaucracy is sclerotic, but when ill-conceived projects go wrong, they blame officials for failing to flag problems earlier, or accuse them of sabotage. Civil servants make easy scapegoats because they cannot speak out in self-defence. That tension is acute when policy springs from ideology and governments resist evidence that might undermine their dogma. Most of what ministers thought they might achieve with Brexit falls into that category. Its implementation was traumatic for the civil service.

That experience also convinced Conservatives that Whitehall was hostile to their agenda – a “blob” that resists reform. Kemi Badenoch has said that about 10% of civil servants are so “obstructive” that they should be in jail. Reform UK approaches state bureaucracy with the same partisan aggression, pledging massive cuts and purges of senior civil servants, who would be replaced by political appointees. Those plans, influenced by Donald Trump’s slash-and-burn approach to independent officialdom, ignore mounting evidence of failure and unforeseen costs caused by the US government’s vandalism of itself.

Some criticisms of the UK civil service are valid, albeit not unique to the public sector: risk aversion, siloed thinking that is closed to innovation, with inadequate reward for success and little accountability for failure, attached to the status quo and lacking in diversity.

Given the growing complexity of demands made of the state and the perennial problem of limited resources, modernisation is necessary. Change is inevitable. But successful reform requires a climate of trust, not fear. It cannot work if civil servants are routinely made scapegoats at times of political stress.

A partnership between reforming politicians and forward-thinking bureaucrats was available when Labour came to office. It looks harder to achieve now, but is no less imperative. The alternative on offer from the opposition is ideological butchery of the state, with devastating consequences for anyone who uses public services or wants a functioning government.

• Do you have an opinion on the issues raised in this article? If you would like to submit a response of up to 300 words by email to be considered for publication in our letters section, please click here.