Survivors group White Flowers Alba said it was seeking a judicial review because the inquiry will not look at their cases.
It claimed this was "unfair" as similar investigations elsewhere in the UK were looking at all instances of abuse.
The Scottish government said the inquiry must focus on a set time frame.
The public inquiry was ordered in December last year and followed allegations which emerged in a BBC Scotland investigation of institutional abuse at a former Catholic boarding school at Fort Augustus in the Highlands.
It is headed by Susan O'Brien QC and will take up to four years to report. It formally began its work last month.
But White Flowers Alba has said its remit was too narrow.
The inquiry does include:
However, it does not include other places where abuse may have happened such as:
Father Gerry Magee, a parish priest in Kilwinning in Ayrshire, is a spokesman for the survivors group.
He said: "It really is like a postcode lottery- if you happen to be in an institution which the inquiry acknowledges and recognises then they will inquire into your abuse but if not, if you were abused in a parish or any other institution - the Scouts or whatever or the church - then you're left out of the inquiry.
"Surely that in itself is a massive injustice."
The Scottish government said the child abuse inquiry was the widest it had ever ordered.
A spokeswoman added: "The initial call was for an inquiry into the abuse of children in institutional care.
"We have listened carefully to survivors of abuse and responded to their request for the scope to be widened.
"This is why the inquiry will now consider instances of the abuse of children in a wide range of care settings. For the inquiry to succeed and reach clear conclusions it needs to focus on an explicit remit within a set time frame."
However, many survivors feel it does not go far enough. They said they had been let down and were now considering legal action.
Andi Lavery, an abuse survivor and spokesman for White Flowers, said: "We should be pushing at an open door on this. Scottish society shouldn't be watching from the sidelines.
"They have to help survivors and prevent further survivors dying instead of leaving us to fight out this endless horrific battle of facing this on our own."
Legal action is being threatened to force the government to widen the remit of the inquiry into historical allegations of child abuse in Scotland.