Buy-to-let investors have faced a stamp duty surcharge, tax relief changes and stricter affordability checks.
The Council of Mortgage Lenders (CML) said an expected recovery in lending to the sector had failed to materialise.
The impact of these changes should be assessed before any new policies were designed, it said.
However, some would argue that the changes have started to rebalance a housing market that had been skewed in favour of buy-to-let investors and had blocked young people from getting on the housing ladder.
The CML had forecast originally that total buy-to-let lending would reach £38bn this year and the same amount in 2018, but it has now cut that to £35bn in 2017 and £33bn in 2018.
Total lending to the sector was nearly £41bn last year.
"Buy-to-let had a weak start to 2017, and the sector's contribution to overall net mortgage lending has fallen considerably over the last year," said CML director general Paul Smee.
"While falling mortgage interest rates have helped support borrowing, tax and prudential measures are exerting pressure on the buy-to-let market. Following the distortion of the stamp duty change on second properties last year, we expected a slight recovery in lending levels. However, this has not materialised, and we therefore have lowered our forecast for buy-to-let lending this year and next.
"This re-emphasises the case for avoiding further changes to the tax and regulatory framework until the effect of these already in train have been properly assessed."
The CML forecast for mortgage lending as a whole stands at £248bn for 2017. However, it said the housing market had "stalled" in the past few months with monthly UK property sales static at around 100,000, reflecting the wider economic picture.
Mortgage lending remained relatively stable with interest rates remaining low - although there are some signs of a slight increase in the offing - and government assistance for helping some first-time buyers.
Where can I afford to live?
Landlords should be spared any new tax and regulation changes after a "weak start" to 2017 for the UK buy-to-let sector, lenders say.