Carbisdale Castle, a former stately home dubbed Castle of Spite because of the part it played in a family feud, was shut two years ago.
The Scottish Youth Hostels Association (SYHA) said the property in Sutherland may not reopen because millions of pounds worth of investment was needed.
The SYHA added that it has plans to open new sites elsewhere.
The organisation also said interest in youth hostelling was at a high.
Keith Legge, SYHA chief executive, said the jewel in association's crown may have had its day.
He said: "Carbisdale has been a great challenge for us financially in recent years.
"We have been very proud to have been running the castle since the last war. 1.5 million guests have stayed there and we are very proud of that record.
"However, maintaining a listed building of that status when it goes wrong presents some very serious challenges. I am afraid to say it won't open for the foreseeable future."
The castle, which overlooks the Kyle of Sutherland, was shut after suffering frost damage and Â£2m has already been spent on repairs.
Historic Scotland lists the castle and its entrance gates as category B, which means they are structures of regional importance.
The Dowager Duchess of Sutherland had Carbisdale Castle built between 1907 and 1917 following the death of her husband, George Granville William Sutherland-Leveson-Gower, the Third Duke of Sutherland.
Lady Mary was the duke's second wife and after he died she became embroiled in a legal dispute over his will with her step son, the fourth Duke of Sutherland.
When the row was settled the duchess used her inheritance to have Carbisdale constructed.
According to geographical encyclopaedia, The Gazetteer of Scotland, the property became known as the Castle of Spite.
Its clock tower only has only three faces with none that could be seen by the new duke as he passed by in his private train on his way to Dunrobin Castle, his family's property in Sutherland. It is said that he would have the blinds of his carriage shut so he did not have to look at Carbisdale.
The castle was bought by a Scots-Norwegian family in 1933 and was used as a refuge for the Norwegian royal family during World War II.
The gazetteer adds that the castle is haunted by a female ghost called Betty.
One of Scotland's most spectacular youth hostels could close for good because of rising repair costs.