The proposal, to be introduced to parliament, would prohibit registered sex offenders from travelling overseas.
Justice Minister Michael Keenan said it would affect about 20,000 offenders who had completed punishments but remained under monitoring by authorities.
Sex offenders would be able to apply for passports if they were no longer on the register, the government said.
"No country has ever taken such decisive and strong action to stop its citizens from going overseas, often to vulnerable countries, to abuse kids," Mr Keenan said.
About 800 registered sex offenders travelled overseas from Australia in 2016, according to the government.
The government said about 3,200 sex offenders would never be eligible for passports because they were being monitored for life.
Mr Keenan described child sex tourism as an "absolutely abhorrent crime".
The proposal was reached with independent Senator Derryn Hinch, long time campaigner for tougher laws to deal with sex offenders.
Mr Hinch said the proposal would protect children.
"You go to Bali, you go to Phnom Penh, you go to Siem Reap, and you see these middle-aged Australian men there, Caucasian men, with a young local kid - they are not there to get a suntan," he told reporters on Tuesday.
Last year, Australian man Robert Andrew Fiddes Ellis was convicted of sexually abusing 11 girls in Indonesia and jailed for 15 years.
Convicted paedophiles would be denied passports in Australia under a "world-first" plan proposed by the government.