Brian Gemmell said a senior MI5 officer told him to stop looking into claims of abuse at Kincora Boys' Home in east Belfast.
He said he presented a report on the allegations to the officer in 1975.
In 1981, three senior care staff at the home were jailed for abusing 11 boys.
It has been claimed that people of the "highest profile" were connected to abuse at the home.
Mr Gemmell, who worked as an intelligence officer in Northern Ireland in the 1970s, has called for a fresh investigation into the home.
He has previously spoken anonymously about his investigations into Kincora, but said he had decided to go public because he feels the allegations need to be investigated again.
The former intelligence officer said that he learned details of what was happening inside the home while gathering information about loyalists.
He said he was told he was running two agents who had close links to Kincora.
However, after presenting his report to a senior MI5 officer, Mr Gemmell said, he was told to cease his investigation.
He said: "I was summoned to go and see him. I went up thinking he was going to be pleased with me.
"He bawled me out. He was rude and offensive and hostile.
"He told me not just to stop any investigation into Kincora, but to drop Royal Flush [an agent he was running]."
Mr Gemmell said Kincora should be investigated again but said "there is not a lot of hope" that it will happen.
"I think there's more hope than there has been in the past. Although there's not a lot, there is more than in the past."
Earlier this month, another former Army officer, Colin Wallace, said any new investigation of Kincora must have access to information from intelligence agencies.
Mr Wallace said he received intelligence in 1973 to say that boys were being abused, but claims some of his superiors refused to pass on the information.
A former army intelligence officer has said he was ordered to stop investigating allegations of child sexual abuse at a boys' home in the 1970s.