EE, O2, Three and Vodafone have agreed to invest Â£5bn and guarantee coverage across 90% of the UK by 2017.
The move should reduce the number of "not-spots", areas of patchy coverage.
The firms had rejected the government's preferred option of a system allowing customers of one network to use another if their supplier wasn't available.
"I am pleased to have secured a legally binding deal with the four mobile networks," said Culture Secretary Sajid Javid.
"Too many parts of the UK regularly suffer from poor mobile coverage leaving them unable to make calls or send texts," he said.
The agreement should reduce total "not-spots", where there is no mobile coverage, by two-thirds, the government said.
Partial "not-spots", areas where there is some coverage but not from all four networks, should be reduced by half, and full coverage from all four operators will increase by 69% to 85% by 2017.
Telecoms regulator Ofcom will enforce the deal.
Olaf Swantee, chief executive of EE, said: "This agreement ensures that our customers are able to stay connected in even more places up and down the country."
A Vodafone UK spokesman said it supported the government's objective of delivering better coverage to rural areas including partial not-spots.
"The voluntary industry commitment we have agreed with the government today will deliver 90% of the UK's land mass with voice services and a major improvement in mobile internet coverage as well.
"It is a great result for UK consumers and businesses and it will make the UK a leader across Europe in terms of the reach of mobile coverage," the spokesman said.
No cash payments will be made by the government to the mobile networks as part of the agreement.
An agreement has been reached between the government and the country's biggest mobile networks to improve mobile coverage across the UK.