The Tories propose changing how rates are calculated, as part of a pledge to put the needs of small businesses at the "heart of the Welsh government".
Welsh Tory leader Andrew RT Davies said small firms were vital to Wales and needed help to "develop and grow".
Retailers warned against "creating winners or losers or more complexity".
The Conservatives would also appoint a team of officials in the Welsh government dedicated to small businesses and answerable to the assembly, if they win power in May's assembly election.
Other measures, some already announced, include scrapping the age limit on a job creation scheme and pledging to make it easier for businesses to borrow money from the Welsh government.
Mr Davies said: "Wales is a nation of shopkeepers, and those small firms are the lifeblood of the Welsh economy; which is why they need support to develop and grow."
Businesses rates are paid to local councils and are based on property values.
There are already discounts available to small businesses from the Welsh government, known as rate relief.
For example, businesses with property worth up to Â£6,000 pay no rates.
The Tories say they would extend rate relief to more businesses.
But they would also introduce different rates for large and small businesses, so that bigger businesses pay more.
England and Scotland already have separate rates, but Wales does not.
The Welsh Retail Consortium, which represents shop owners,  called for "fundamental reform of business rates for all, whether that be small, medium sized or larger businesses".
A spokeswoman said: "The proposed small firms' relief is a welcome acknowledgement of the need to keep down costs, however many high street retailers won't benefit and we would be alarmed if this new rate for medium and larger sized commercial premises simply opened the door to even higher taxes, as has been proposed in Scotland."
Supermarkets and other big businesses could face higher business rates under a Welsh Conservative government, with a tax cut for smaller companies.