The restricted compartments of the UK’s national security infrastructure are clearly defined and closely controlled. To work across them requires “a developed vetting certificate”. The primary qualification for holding a “DV” is integrity, honesty and transparency in one’s personal and professional life. To lie about or hide potential vulnerabilities is an immediate disqualification. Staff who do not meet the DV requirements for whatever reason are barred from positions that demand DV clearance. There are no grey areas or soft edges.

The role of British ambassador in Washington is one of those posts. It sits across a number of highly classified compartments. It is no ordinary diplomatic job. The extensive security acreage of the special relationship includes, for example, the UK’s nuclear deterrent, the intelligence relationship, the UK-US alliance which ties together the National Security Agency and GCHQ by treaty, and other domains of great sensitivity. The ambassador has access to these even though their need to become involved in them in normal times is limited. The British staff that comes under the ambassador’s authority is extensive and stretches beyond those working in the embassy. The ambassador’s access to the US administration is also usually highly privileged, such is the nature of the special relationship.

Recent British ambassadors to the US have been professional diplomats. They would have been fully accustomed to the discipline of the DV process. A political appointee would be less familiar, though a former minister would have knowledge of it. The process depends on a personal interview, the interviewing of referees and a degree of investigation: tracing criminal records and more extensive inquiries where concerns about an individual may have arisen. Departments with large numbers of DV’d staff have their own vetting team, often made up of experienced retirees. In Peter Mandelson’s case he was vetted by a central vetting department attached to the Cabinet Office.

What is extraordinary about Mandelson’s appointment is that it was announced before he was vetted, without any qualification that it would be subject to security clearance; and we know that the cabinet secretary had alerted the prime minister to this risk. With the benefit of hindsight it is scarcely credible he should even have been considered for a high-profile DV’d post. His professional life was heavily freighted with questions over his integrity. The ministerial resignation scandals are a matter of public record, but there were also his dealings with various Russians, for example the tycoon Oleg Deripaska (remember the photos on the yacht) and the entanglement of his global consultancy with leading figures in the People’s Republic of China, let alone the known fact of his relationship with Epstein. The choice of Mandelson, despite his starry political record as the talisman of New Labour, was from the very start deeply flawed.

One can only conclude that the prime minister, in his enthusiasm to appoint New Labour’s “uber fixer” as his Trump whisperer, failed to take account of the inherent risk – as did the PM’s advisers.

The decision to appoint Mandelson was called into question when the vetters recommended against granting the DV; but the new Foreign Office permanent secretary, Olly Robbins, chose not to oppose Mandelson going to Washington. He was under heavy pressure to approve the appointment so understandably decided that the risk could be managed rather than alerting his minister, the foreign secretary, to the problem.

My own experience with vetting problems of this type (though without the explosive political charge of this particular one) was never to allow anyone the benefit of a doubt. The permanent secretary therefore paid a heavy price for not unloading the problem on to his political masters. Clearly that was a misjudgment, but either way his own position was probably at risk: oppose the PM’s wish or manipulate the process with mandarinate dexterity. Either choice was potentially career-ending, as has proved to be the case.

The political wrecking ball that these events has eventually set swinging has demolished the Mandelson edifice. Overdue, but the house of Starmer too is tottering on shallow foundations. An important quality we expect from our prime ministers is good judgment. The office has been suffering from a deficit of it, not just restricted to the present incumbent, a malaise which voters now well understand. A confident prediction is that the political fallout is going to be profound.

• Richard Dearlove is the former head of the British Secret Intelligence Service

• Do you have an opinion on the issues raised in this article? If you would like to submit a response of up to 300 words by email to be considered for publication in our letters section, please click here.