Anthony Albanese has accused Angus Taylor of copying One Nation and “dog-whistling” to marginalise immigrants to Australia, as the Coalition seeks to cut social benefits and government programs for permanent residents.

Delivering his budget reply speech on Thursday night, the opposition leader announced plans to slash immigration and restrict welfare programs to citizens only.

This would cut access to the national disability insurance scheme, jobseeker and other safety nets from permanent residents who could be living in Australia and paying tax for years.

There are already years-long waiting periods for non-citizens to access such payments, meaning the plan may have relatively little impact.

Labor on Friday savaged Taylor for taking aim at migrants after previous plans to restrict first home buyer grants to citizens.

The prime minister claimed the Coalition was imitating One Nation’s hardline anti-immigrant stance.

“This is the tail wagging the Coalition dog – and dog is an appropriate term, because a lot of the speech was about dog-whistling,” Albanese said.

“I wonder what the distinction between Australians and migrants is, because I’ve never seen that used before in a country like ours.

“We have people who are in senior positions in our government, in business, some of the giants of business in this country came here as refugees and as migrants. Are they separate from Australians?”

Taylor defended his stance on Friday, saying new migrants “automatically get access to some welfare benefits”, which he called “completely inappropriate”.

But on ABC radio, Taylor was factchecked by host Melissa Clarke, who pointed out the jobseeker payment had a four-year waiting period, while the age pension and disability support pension had a 10-year waiting period.

Taylor responded: “Sure, but citizenship has got to mean something. We want people to aspire to be citizens of Australia.”

Gaining Australian citizenship requires someone to have been living in Australia on a valid visa for at least four years immediately before the day they apply, meaning even aspiring citizens who begin their application as soon as possible could be without help for at least four years.

“We firmly believe that the government’s priorities are wrong,” Taylor continued.

“At exactly the same time, they’ve ripped money away from pensioners and from older Australians with support for their private health insurance.”

Taylor pledged to address what he called “mass migration madness”, including a plan to link Australia’s migration intake to the amount of new housing built.

It came after his party’s vote crashed in the long-held seat of Farrer, which was won by One Nation last weekend. The Liberal candidate polled only 12% of the primary vote – compared with Sussan Ley’s 43% just a year prior.

Against a surging One Nation that has been outperforming the Coalition in most major polls, Taylor’s opposition has been accused by the government of copying Pauline Hanson’s policies.

Migrant groups condemned the Coalition’s policy as “a dangerous escalation of dog-whistle politics that targets communities of colour” rather than a serious answer to the housing crisis.

Noura Mansour, the national director of Democracy in Colour, said the proposal was a transparent attempt to use race and migration as a political wedge.

“We are not in Trump’s America,” Mansour said on Friday.

“Angus Taylor’s proposal is a race to the bottom with the far right. By suggesting that we need to ‘discriminate based on values’, the Coalition is creating a two-tiered society where your right to belong is constantly under threat based on the whims of a politician.”

A spokesperson for the Refugee Council of Australia raised an alarm about the Coalition’s rhetoric.

“Australia needs leaders who are aiming to bring people together, not tear them apart. Migrants, refugees and people seeking asylum should not be used as scapegoats as Australia tries to resolve policy failures in housing, cost of living or access to services.”

“Most people who have migrated to Australia want to become citizens but it is a process which takes years. Proposals to exclude and penalise non-citizens only deepen division and unfairly target people who are already contributing to Australian society.”

The national chair of the Australia China Business Council, Rhys Roberts, would not weigh in on the political debate around migration but called for the discussion to remain respectful.

“Public discussion around migration, the economy and social policy should remain respectful, fact-based and focused on practical outcomes that support social cohesion, economic resilience and Australia’s long-term prosperity,” Roberts said.

He said Chinese migrants and businesses had “made a significant and enduring contribution” to the country since the 1850s.

“Today, Chinese-Australian businesses continue to contribute across a wide range of sectors.”

The treasurer, Jim Chalmers, called Taylor’s budget reply “a ploy to stave off One Nation”, accusing the opposition leader of whipping up “more division in this country”.

The housing minister, Clare O’Neil, claimed the Liberals, Nationals and One Nation “want to defend the status quo, point the finger at migrants, and say this is everyone else’s fault”.

“What I would say to Angus Taylor is that you can’t out-One Nation One Nation. If people like what Pauline Hanson is putting down, they’re going to vote for them, not for you,” O’Neil said on Thursday.

Hanson on Thursday claimed Taylor was “on borrowed time with borrowed policies”, accusing him of “duplicating” her policies for his budget reply.

Taylor was contacted for comment.