Three planning committee members declared an interest in the project and took no part in the debate or vote, reducing the committee to four members.
Last week a petition to Scottish Borders Council failed to get it to drop its financial backing of the plan.
Now it has approved proposals for a two-storey building near at Tweedbank.
It will be sited near to the terminus of the Borders Railway.
The tapestry will be displayed on the upper floor with a cafe, shop and toilets on the ground floor.
Those behind the project believe it will attract many visitors to Tweedbank and will be an asset to the wider Borders tourism industry.
The scheme attracted 74 objections and seven letters of support.
Those against it claimed there were a number of alternative locations in towns like Galashiels, Hawick and Selkirk, which could probably offer more for visitors.
They also believed the loss of trees on the Tweedbank site would impact on local residents and wildlife.
Those in favour said it was a special site for a special building and that the only way to do the tapestry justice was to house it in a purpose-built exhibition centre.
Planning officials recommended that the application should be approved but they urged councillors to attach a number of conditions regarding landscaping, drainage, parking and wildlife protection.
Get live news updates from the Borders and Dumfries and Galloway throughout the day on our South of Scotland live page service.
Councillors have unanimously approved an application for a £6m permanent home for the Great Tapestry of Scotland at Tweedbank.