Ruling on an issue of great interest to prosecutors, civil libertarians and the police, New York State's highest court has unanimously approved the use of hidden television cameras in criminal investigations, even though such surveillance is not specifically allowed by any state law.

The issue, which involves questions of Federal constitutional and statutory law, ultimately will have to be resolved by the United States Supreme Court.

The New York Court of Appeals did not limit its ruling to office or business premises, leaving open the possibility that the police could legally install hidden cameras in the home of a criminal suspect.

In a decision announced Tuesday, the court held that if the police obtained a search warrant in advance, they could install hidden videotape surveillance cameras in a location where there is probable cause to believe a crime is going to be committed.

But it also called on the State Legislature to enact specific guidelines regulating television surveillance ''because the use of this investigative technique poses a threat to the privacy of citizens.'' Ruling in Dentist's Case

The Court of Appeals made its ruling in upholding the 1978 conviction and four-month prison sentence of Dr. Marvin Teicher, a Manhattan dentist, for sexually abusing one of his female patients and an undercover female detective posing as a patient.

Dr. Teicher's lawyer, Jacob W. Heller of Manhattan, said he would file a further appeal to the United States Supreme Court. On Wednesday, Judge Jacob D. Fuchsberg of the State Court of Appeals granted a temporary stay canceling an earlier order that Dr. Teicher begin serving his prison sentence next Tuesday.

In 1976, relying on a court warrant for which there was no precedent in state law, New York City detectives broke into Dr. Teicher's office and installed a concealed, closed-circuit television camera inside the drop ceiling of the examining room. On July 27, an undercover detective who needed a wisdom tooth extracted was rendered unconscious by Dr. Teicher with a sedative injection.

The police videotaped what took place in the examining room with a monitor connected by coaxial cable to the hidden camera, and the videotape was later used as evidence at the dentist's trial. The Difference in Abscam

Although hidden videotape cameras were used in the widely publicized Abscam cases, that surveillance was conducted without court warrants under a long-established legal rule that allows ''consensual or participatory monitoring'' if at least one of the persons involved - usually a detective - consents.

What made Dr. Teicher's surveillance legally different was that the detective posing as a patient was unconscious and thus could not technically consent to the television monitoring as an active participant. The intrusion of the hidden camera thus had to be considered on its own merits, just as a hidden microphone would in a case of aural surveillance.

Mr. Heller, the defense attorney, had contended, among other things, that television surveillance was such a dangerous intrusion of privacy that it should be ruled illegal under the Fourth Amendment.

The Court of Appeals, however, rejected all the defense arguments. In an opinion written by Judge Domenick L. Gabrielli, it ruled that ''the Fourth Amendment does not mandate an absolute ban on video surveillance any more than it mandates a total proscription on electronic eavesdropping.'' Reason for Validity

In Dr. Teicher's case, the court held, the search warrant authorizing the television surveillance was valid under a state law allowing ''personal property'' to be seized with a warrant. Such ''personal property,'' Judge Gabrielli wrote, was not limited to the seizure of tangible objects and could be interpreted to include the seizure by a videotape camera of intangible visual images.

Although Federal wiretapping laws pre-empt conflicting state laws, Judge Gabrielli said they did not invalidate the New York statute because they applied only to aural interceptions and were silent on television surveillance.

But he added that because of the ''Orwellian overtones'' of television surveillance, the following ''safeguards'' would have to be observed with ''unswerving adherence'' in future cases:

- The judge who is asked to issue a television surveillance warrant must be shown probable cause to believe that a particular person is committing, has committed or is about to commit a crime in the place which is to be the object of the surveillance.

- The warrant must describe precisely the crime under investigation, the type of activity sought to be observed with the camera and the person or persons expected to be observed conducting the illegal acts.

- The extent of the surveillance must be kept to a minimum, and the observation and videotaping must be limited to the activities for which the warrant was issued.

- Before the warrant is issued, the police must show that there are no less intrusive investigatory means available for obtaining the necessary incriminating evidence. Morgenthau Hails Ruling

District Attorney Robert M. Morgenthau of Manhattan, whose staff prosecuted Dr. Teicher, welcomed the court's ruling. ''I think it will be an important tool for law enforcement to be able to install a videotape camera pursuant to a warrant,'' he said.

''It will be helpful particularly in the one-on-one situation where the only persons present are the perpetrator and the victim and you need the corroboration. I don't see it as a widespread investigative device, but I think it will be used in appropriate circumstances.''

But Dorothy J. Samuels, executive director of the New York Civil Liberties Union, reacted to the decision with concern. ''There's no question,'' she said, ''that granting police broad discretion to break into people's offices and homes to install hidden television cameras brings us one step closer to 1984.''