Ms Gillard had presented redrafted legislation to the opposition as she sought support for her "Malaysia swap".
But the opposition said the proposals did not provide adequate protection for asylum-seekers.
Australia's asylum policy has been in crisis since a court ruled a plan to swap refugees with Malaysia unlawful.
Australia detains all those who arrive by boat, some on the mainland and some at the Christmas Island detention centre.
The government says its "Malaysia swap" plan aims to deter people from making the dangerous sea voyage.
Under the plan, Australia would have sent 800 asylum-seekers who arrived by boat on to Malaysia and would have received 4,000 refugees in return over four years.
But the court ruled Malaysia - which has not signed UN refugee conventions - did not offer adequate protection.
The ruling also threw plans to send asylum-seekers to re-opened processing camps in Papua New Guinea or on the Pacific island of Nauru in doubt.
Ms Gillard leads a minority government which depends on the Greens for support. The Greens do not support offshore processing, forcing Labor to look to the opposition for co-operation.
Source: Australian Department of Immigration
She wants to change the Migration Act to allow Australia to send asylum-seekers to Malaysia.
But opposition leader Tony Abbott said the proposed changes paid "lip service to protections without actually guaranteeing them", and said his party had rejected them.
Instead, he said his party would present an alternative proposal which excludes a deal with Malaysia.
The Liberals want to reopen a processing centre on the Pacific island of Nauru - a policy that was heavily criticised by the Labor party.
Immigration Minister Chris Bowen said Labor would "not be going down that road".
"Very clearly, if this legislation doesn't pass, then offshore processing will be regarded as unlawful," The Australian newspaper quoted him as saying.
"And therefore the obvious result of that is onshore processing."
Last year 6,535 asylum-seekers arrived in Australia by boat.
Both parties have sought to appear tough on asylum, despite recent opinions polls which suggest many Australians support onshore processing of asylum-seekers.
Australia's opposition rejected the government's proposed immigration law changes, as PM Julia Gillard scrambled to salvage a controversial asylum plan.