East Timor wants the court to decide who owns a large undersea oil and gas field.
East Timor says a 2006 treaty for revenue sharing was unfairly forced upon the newly independent nation.
Australia says the international court has no jurisdiction over the dispute.
On Monday, the Permanent Court of Arbitration (PCA) "held that it was competent to continue with the conciliation process" brought by East Timor against Australia in April.
The Greater Sunrise oil and gas field lies between the two nations.
East Timor - also known as Timor Leste - welcomed the decision by the PCA - the world's oldest arbitration tribunal.
"Just as we fought so hard and suffered so much for our independence, Timor-Leste will not rest until we have our sovereign rights over both land and sea," said Xanana Gusmao, the independence resistance hero and former prime minister.
Australia's Foreign Affairs Minister Julie Bishop said her country accepted the decision.
"We are committed to working together to strengthen our relationship and overcome our differences in the Timor sea," she said.
The international court of arbitration in the Netherlands has agreed to take up a decade-long maritime boundary dispute case between Australia and East Timor.