It said landline prices had risen by up to 41% in real terms since 2010, while wholesale prices have fallen by a quarter.
The rise in rental prices particularly affects those who rely on landlines, such as the elderly, Ofcom said.
The review will decide whether measures are needed to protect them.
The worst offenders are Sky, which has raised landline prices by 41% in real terms over the past six years, and Virgin Media, which has raised prices by 38%.
In response, Virgin Media said it would freeze landline charges for its elderly and disabled customers from January 2017.
Jonathan Oxley, Ofcom's competition group director, said: "Our evidence shows that landline providers have been raising the price of line rental, even as their costs have been coming down.
"We're particularly concerned for older and vulnerable customers, who rely on their landline and are less likely to change provider. So we're reviewing this market to ensure these customers are protected and getting value for money."
The most expensive provider for landline-only services is currently Virgin Media, at Â£19 a month, followed by BT at Â£18.99, according to Ofcom.
All in all it's thought around 10% of households in the UK do not have broadband or television packages.
As a result some two million homes rely on landlines or mobiles. Around a million are completely dependent on landlines.
Digital minister Matt Hancock said, "It cannot be right that these customers are paying over the odds and I am pleased that Ofcom is taking action on this important issue.
"The government is clear that action should be taken where consumers are not getting a good deal, which is why we are launching a Green Paper next year to examine markets which are not working fairly."
Ofcom has been tracking landline rental prices for some time, and has concerns that competition is not working as effectively as it could in the market, an Ofcom spokeswoman said.
Its review will look at whether any one provider has significant power, she said.
Ofcom does not have competition concerns about the market for bundling services such as broadband, pay-TV and landline rental together.
Telecoms regulator Ofcom has launched a review of landline-only rental prices after concerns that people may not be getting value for money.