Devon and Cornwall Police are now investigating the claims made against Exeter Deaf Academy.
The BBC understands three staff members have been suspended, but the school would not confirm this.
The academy said it would act "quickly if it is found that any improvements need to be made".
More updates on the claims, and other Devon news here
Sources from the inquiry told the BBC they received a letter from the whistleblower detailing claims which included staff drinking alcohol on residential trips.
It is claimed the drinking led to a lack of supervision of students who engaged in sexual activity.
The letter also alleges excessive force was used to restrain children, along with techniques that were not authorised by regulators.
The concerns were raised, the letter says, but not dealt with.
The allegations prompted Devon and Cornwall Police and Devon Social Services to launch an investigation.
Exeter Deaf Academy said: "We continue to cooperate fully with the ongoing investigation by Devon County Council and Devon and Cornwall Police and cannot and should not comment until this is complete and the facts have been properly established.
"We care deeply about the welfare of our students, and will act quickly if it is found that any improvements need to be made at the Academy."
The academy - a mixed, residential facility that currently has 76 students aged between four and 24 -  would not comment on whether or not any staff had been suspended.
Devon and Cornwall Police said: "We are not yet in a position to comment regarding whether there have been any criminal offences or other breaches of legislation, however, the safety of the students is paramount and we will work together to ensure that a thorough investigation is conducted."
Devon County Council said it was "working closely" with the police and the academy's chief executive.
The latest report by the regulator Ofsted, in February this year, was critical of the academy saying it "requires improvement".
Children at a school for the deaf were mishandled by staff, allowed to misbehave and engaged in sexual activity, a whistleblower has said.