When you think of retirement, what comes to mind? Perhaps it is images of older people enjoying a well-deserved period of leisure and comfort in the final stretch of their lives. Cruise ships, garden centres, golf clubs and bungalows by the sea. The truth is that this image is now, in large part, the artefact of a bygone age. A long and comfortable retirement starting at 60 or 65 is beginning to look like a collective social experience whose moment has passed. The political and economic forces it relied upon appear to have run their course – and it’s time to start thinking about what comes next.

Retirement in Britain has a surprisingly short history, underpinned by dramatic improvements in older people’s quality of life over the past 50 years. Large public and private bureaucracies first started to enrol long-serving employees into pension schemes from the mid-19th century. In 1909, Britain was the first country to pioneer an old age pension, funded by the state and targeting the poorest, who could claim it from the age of 70. But it was only after the second world war that a period of leisured old age become an ordinary expectation for most British workers.

These aspirational lifestyles had many sources. The state pension had been made universal by Clement Attlee’s Labour government, alongside expanding occupational pension schemes and rising home ownership. In the 1960s, retirees acquired a taste for travel through the explosion in cheap package holidays, and in the 1970s and early 1980s embraced lifelong learning by joining such bodies as the Open University and the University of the Third Age. As healthy life expectancy rose against a backdrop of full employment, high wages and free NHS care, older Britons enjoyed earlier and more active retirements. Gone were the days of working until your body gave out simply to make ends meet.

Not everything was golden for the over-60s. As economic conditions worsened from the 1970s, some workers in declining industries felt moral pressure to accept redundancy and preserve jobs for the young. Income inequalities deepened during the 1980s as Conservative governments allowed the value of the state pension to fall and encouraged individuals to build private pension pots and invest them in volatile global equity markets. When these performed well, the rewards could be substantial. But so were the risks, a fact brought home by the financial crisis of 2007-8, which saw the value of pension funds plunge.

Yet viewed in the aggregate and over the longer term, older Britons became steadily wealthier. When the large postwar cohorts of baby boomers began to retire around the millennium, they formed the richest, fittest and best-educated generation of retirees yet. Those with inflation-proofed, final-salary pensions could exercise unprecedented choice over retiring early or staying in work, the latter path enabled by age-friendly employers and flexible modes of self-employment. This contributed to a movement away from ever-earlier retirements in the 2000s.

Meanwhile, those with weaker pension rights, including many women and ethnic-minority citizens, alongside disabled and chronically sick people, benefited from New Labour’s minimum income guarantee, winter fuel allowance and free television licences. In 2003, for the first time in postwar history, the proportion of pensioners experiencing relative poverty dropped below the national average.

Older people’s demands for better retirements have forced successive governments to act, and kept these issues on the agenda. Pensioner organisations campaigned for more generous treatment since the 1930s, but only in the later 20th century did they become powerful advocacy bodies. One major voice was Age Concern, which merged with Help the Aged to become Age UK in 2010. Another was the National Pensioners Convention, founded in 1979 and with close links to the trade union movement. Both worked relentlessly to keep older people’s needs in the public spotlight and at the forefront of ministerial agendas.

Moreover, when retirement dreams were imperilled, Britons were prepared to fight back, as exemplified by the angry Mirror pensioners ripped off by Robert Maxwell’s crookery in the early 1990s, or the Waspi women who continue to press for financial compensation following the equalisation of men and women’s state pension age.

Emboldened by the new language of “ageism” and, from 2006, legal protections against age discrimination, the over-60s argued for their rights. With strength in numbers at the ballot box, they discovered that they did not have to settle for less, simply because they were old.

All of this represents a remarkable social achievement. While many still experience financial hardship, the indignity of having to scrape by to keep yourself warm, clothed and fed in old age has diminished dramatically since the mid-20th century. Yet this historic improvement has attracted less celebration than might be expected. In recent times, the politics of ageing has taken a new turn.

In the aftermath of the financial crash and under the sword of the coalition government’s spending cuts, debates on the economy and welfare state sharpened around questions of intergenerational fairness. Ageing boomers were reimagined in books such as David Willetts’s The Pinch: How the Baby Boomers Took Their Children’s Future and Why They Should Give It Back as a problem generation whose selfishness was destabilising public finances and fuelling social conflict. A narrative took hold in which younger Britons, saddled with student debt, dogged by sluggish wage growth and locked out of home ownership, were suffering in order to preserve the triple-locked state pension and boomer assets. With no guarantee of a similarly comfortable retirement for themselves, the generational contract was broken. The Brexit referendum and Covid pandemic drove the wedge deeper.

Much was left out of this account, not least the inequalities existing within generations, as well as the cross-generational solidarities of family. Nonetheless, governments need to wake up to the fact that generational politics will play a big part in shaping the future of retirement. When generation X, now in their 40s and 50s, begin to retire, pensioner incomes are likely to fall, drawing to a close the chapter that has seen every postwar cohort enjoy greater security in later life than the one that went before.

This is because, as recent research by the Social Market Foundation reveals, generation X entered the labour market at a moment when generous defined- benefit schemes were being replaced by defined contribution, which were less costly for employers and riskier for workers. As a result, retirees of the 2030s and 40s will have smaller pension pots than the boomers, although they will leave work with more housing wealth than the millennials and gen Z coming behind them. Since 2012 firms have been obliged to enrol employees in pension schemes, meaning that those younger cohorts are now saving. But nowhere near enough.

In short, we stand on the brink of a renewed era of old-age austerity. Much rests on how Britons respond. Some are already embracing the so-called Fire movement, reducing their consumption in the hope that they can achieve Financial Independence, Retire Early. Others seem resigned to the prospect of working late into their 60s or 70s, forgoing the sunset years of leisure enjoyed by their parents and grandparents.

Can anything be done to keep the dream of a comfortable retirement alive for future generations? Pensions reform might safeguard basic living standards, but in the longer term our ideas about later life need to change. There must be no return to the days when Britons retired into poverty after decades of hard manual labour. Instead, we should rethink how to blend work, care, learning and leisure across the life course, harnessing technologies and embracing ways of being that sustain rather than harm our planet. The right to retire was yesterday’s struggle. Today’s is the right to live a good, meaningful life, and to live it right to the end.

• Helen McCarthy is a historian and the author of Double Lives: A History of Working Motherhood