The City watchdog, the Financial Conduct Authority (FCA), raised concerns about the clarity of fees and the treatment of customers in arrears.
It also questioned the way customers were given payment meters to restrict access to TVs.
Dunraven Finance will pay Â£939,000 in compensation.
The company, which uses the trading name Buy as You View, will pay an average of nearly Â£16 per customer, by adjusting bills or in cash payments. It will contact customers directly.
Chief executive Graham Clarke said: "We have worked closely with the FCA in recent months to address these issues and I am sorry to any of our customers who may have experienced difficulties as a result of us not achieving the high standards we set ourselves."
Instead of the current policy of restricting access to the customer's television when payments are not received on time, the company will issue a default notice at least 14 days prior to restrictions.
It is also conducting a review of customers who may have been offered more credit than they should have been.
The vast majority of customers to receive compensation will do so as a result of fees applied to their accounts for unpaid direct debits since 2001.
Jonathan Davidson, from the FCA, said: "It is important that firms meet our standards, including carrying out proper creditworthiness assessment and making sure that those in difficulty are treated fairly.
"We will continue, when necessary, to take action against inappropriate behaviour."
Compensation is to be paid to 59,000 customers of Buy as You View, which rents out furniture, TVs and other electronic goods on hire purchase.