Nick Webb said a mayor serving Cardiff and Newport and another for the Swansea Bay area would be more effective than their local councils.
He said elected mayors in England were controversial, but allowed voters to "kick out those who fail to deliver".
Writing for the Institute of Welsh Affairs, he said the mayoral regions could expand if they proved successful.
The UK government is encouraging cities to bid for powers and funding for big economic projects under schemes such as city deals and the Northern Powerhouse.
Ten local councils in south east Wales are backing a bid for a Â£1bn-plus "city deal" centred on Cardiff.
Further west, four councils and local business leaders have joined forces to seek investment as a Swansea Bay city region.
But Mr Webb argued that while English city regions were set to be run by "powerful independent-thinking people", Welsh city regions were "poorly defined" quangos led by appointed chairmen.
"If the elected Mayors of London, Manchester, Birmingham, Sheffield, Leeds, Liverpool and Newcastle are sitting around the table in Davos talking to international financiers, it is hard to see that a Welsh Government-appointed City Region Chair - or a combination of local authority leaders - would hold the same powerful mandate when pitching their case," he said.
Mr Webb, who stood for the Conservatives in Newport West at the general election and previously for Gwent police and crime commissioner, said neighbouring councils should be allowed to join the region served by an elected mayor if they saw benefits from it.
In March, a study about the impact of Bristol's elected mayor George Ferguson reported "a dramatic increase in the visibility of city leadership" but mixed views on whether the post had improved governance.
Elected mayors could give Welsh cities "more clout" and make them more accountable, a Welsh Tory has said.