Meanwhile, the Archbishop of Canterbury has said the Church could do more to help non-profit lenders to compete with payday firms.
The Most Rev Justin Welby wants to see skills of members of the congregation, as well as Church premises, used to assist the advance of credit unions.
So how do these institutions work, and how can people borrow from them?
What are credit unions?
They are financial co-operatives, owned by the people that use them.
Generally, membership of an individual union is limited to people who live and work in the local area it serves.
They may also be attached to a workplace - so only people who work there can join - or a trade union, a religious group or a housing association.
How do I save at a credit union?
Members are encouraged, first and foremost, to save rather than borrow.
They can pay the money in at local offices, collection points, some newsagents, directly from wages, or through a standing order or direct debit.
These savings are protected, up to £85,000, by the Financial Services Compensation Scheme, were the credit union to go bust.
And how do I borrow?
Credit unions can lend money to members, but the amount they can lend does vary.
Some will only lend up to £1,000, but some of the bigger ones may offer larger loans or even mortgages.
They are set up to offer loans at affordable rates, so can only charge a maximum of 2% a month (26.8% APR). That means a £500 loan repaid over six months will cost no more than £36 in interest.
That sounds a lot less than some payday lenders?
It is. However, that brings some of its own issues.
It means that credit unions are unlikely to have the scale of many payday lenders, and so could struggle to compete with the hi-tech websites and speedy applications that payday lenders offer.
As a result, the government wants to extend the interest that credit unions can charge to 3% a month (42.6% APR).
What will be the result of that?
The government hopes that, alongside £36m in extra funding, the membership of credit unions will double to two million.
There are about 400 credit unions in England, Scotland and Wales at present.
How do I find my nearest credit union?
Residents can find their nearest credit union through an online search provided by the Association of British Credit Unions.
The government wants to double the membership of credit unions to challenge the growth of the payday lending sector.