Angus Taylor has refused to apologise for calling Anthony Albanese an 'arrogant p***k', claiming Australians are saying 'far worse' about the PM.

The Opposition Leader and the Prime Minister came to blows during Question Time on Thursday while discussing Labor's tax changes in the Federal Budget this month. 

Taylor muttered the insult after Albanese answered a question about changes to the capital gains tax discount and negative gearing.

His words were quickly reported by Tony Burke, the Minister for Home Affairs, who told Speaker Milton Dick they were 'way beyond unparliamentary'.

Several Liberal and Labor MPs later confirmed hearing Taylor's comment. 

Speaker Milton Dick said he did not hear the insult but asked the Opposition leader to withdraw the 'unparliamentary remark', which he did. 

On Friday, Taylor defended his words, telling Today show host Karl Stefanovic that Australians are saying 'far worse' about Albanese.  

'I have to say that I'm hearing far worse at the moment from others as I get around the country. Small business people and farmers, they're angry,' he said. 

Stefanovic asked if Taylor was sorry or 'regretted' his words. 

'I had to withdraw it. It was unparliamentary. But what I am hearing everywhere I go is far worse than that, Karl,' the Opposition leader replied. 

'If you're trying to grow a business, get ahead, create opportunity for yourself and your family and your community, you're going to get punished by this government.

'They (the Albanese government) have lied to them all the way along. And the worst thing, Karl, is that we ask questions of the Prime Minister in Question Time. 

'He doesn't even try to answer them. He's not even attempting. I don't think he understands it or he's trying to get around it.'

Stefanovic said it was a side of Taylor that Australians 'don't often see', adding: 'So we might just see a little bit more of that.'

Taylor's discussion with Stefanovic came as Master Builders Australia, the Property Council of Australia and the Real Estate Institute of Australia released new modelling on Friday highlighting the Budget's impact on the housing market.

'The overall impact of the Federal Budget will see new home construction go backwards, and higher than anticipated rental inflation,' the release said. 

Over the next four years, it will cause new housing supply to fall by over 8,700, rents to increase by up to $9 per week, a reduction in GDP of $864million. 

Additionally, construction jobs are forecast to fall by more than 3,800.

'On a $600 per week rental, the increase to rents equates to $142 per year in 2026-27, rising to $477 per year in 2029-30,' the release said. 

Taylor used the modelling to take aim at the Albanese government for its 'tax grab' during the interview. 

'In an uncharacteristic piece of honesty in the Budget papers, the government admitted that their toxic taxes were going to reduce the supply of housing and increase rents and this modelling out today confirms exactly that,' he said.

'Housing construction has gone backwards. So no wonder young Australians can't get into a home. These taxes are only going to make the situation worse. 

'It is an abject failure from this government.'

Defending his government's tax measures during Question Time, Albanese suggested the Coalition was more upset by lowered income tax.

'The leader of the Opposition was triggered when I was talking about reducing income taxes,' he said. 'Immediately you speak about reducing taxes and they get upset. They've never been more angry.'

Albanese also pointed out that the Opposition had opposed many of the tax cuts it had previously introduced, such as the Stage 3 tax cuts in 2024.

Labor's bill was automatically referred to a Senate committee for scrutiny under rules requiring laws effective from July 1 to be reviewed before taking effect.

The committee will need to report back by June 22.