Anthony Albanese has declared he finds the barrage of betting advertisements during sporting matches “annoying” after opposition leader Peter Dutton proposed a ban because “footy time is family time”.

In an interview with Guardian Australian, the prime minister said he would not directly comment on any plans to ban this advertising, saying there was a review under way.

• Sign up for Guardian Australia’s free morning and afternoonemail newsletters for your daily news roundup

But asked whether all regulatory roads now led to a ban on gambling advertisements during sporting matches given rising community concern about the ubiquity of wagering messages, Albanese said: “I don’t want to pre-empt the review which is under way.”

“But on a personal level, I find them annoying,” he said.

Dutton used his budget reply speech last week to float a ban on sports betting advertising during broadcasts and for an hour each side of a sporting game.

Joining calls for action from the Greens and independents, Dutton told parliament last Thursday the “bombardment” of gaming advertisements took the “joy” out of televised sports. “Worse, they are changing the culture of our country in a bad way and normalising gambling at a young age,” the Liberal leader said.

Albanese said Dutton’s advocacy of the ban was surprising given the Coalition “did nothing for nine years on any of those issues”.

He said the government had been proactive in policy terms. “We’ve been in government for a year and we’ve got a review … that we initiated after we after we’ve already changed the advertising guidelines and strengthened them regarding any advertising for gambling”.

The prime minister noted the government had pursued new mandatory harm minimisation messages on advertisements, banned the use of credit cards for online wagering, and restricted gambling-like activity in video games.

02:46

Any advertising ban ultimately enacted by the parliament would face a significant backlash from the gaming sector. The peak body for commercial television has already expressed objections, with industry estimates suggesting broadcasters could lose hundreds of millions in advertising revenue.

Calls to ban wagering ads during sporting broadcasts have been slammed by the peak body for commercial TV. Free TV has said the sector was already taking action.

Albanese’s signal on the advertising ban in the post-budget wash up came as the treasurer Jim Chalmers on Sunday also kept open the option of taking up the opposition’s proposal to increase the hours job seekers can work before losing their payments.

Dutton withheld support for the government’s proposed $40 a fortnight increase to the jobseeker rate in his budget reply speech on Thursday, calling instead for welfare recipients to be able to earn more before payments are reduced.

But Albanese rebuked Dutton’s negative politicking on immigration.

Last Thursday the opposition leader blasted Labor for the fact net migration was projected to increase “massively by 1.5 million people over five years”.

“The Albanese government’s big Australia approach will make the cost-of-living crisis and inflation worse,” Dutton told parliament.

Albanese said politicians had an obligation to act responsibly and keep debates factual.

After neo-Nazi and anti-fascist groups clashed in Melbourne again on Saturday, and Joe Biden warned over the weekend that white supremacy was currently the “most dangerous terrorist threat” to the United States, Albanese said Australians were tolerant and respectful people.

But the prime minister noted Australia’s domestic security agency had issued public warnings about rising extremism. “I think debate has to be factual,” he said.

“It is legitimate for people to raise issues, but to mischaracterise the temporary increase in migration as a result of the pandemic ending and the opening up of borders – I think it is important that political leaders stick to facts.”