Leon Briggs, 38, from Luton, died after being held following concerns about his behaviour in the town on Monday.
The Independent Police Complaints Commission (IPCC) said it believed "that potential criminal offences may have been committed".
The offences could include negligence or manslaughter, it said.
IPCC Commissioner Mary Cunneen said: "Following a review of the evidence gathered so far, I have determined that this will be a criminal investigation. We are still in the early stages of this investigation and it is important that we do not pre-judge the findings.
"However, at this stage we believe there is an indication that potential criminal offences may have been committed including gross negligence and/or unlawful act manslaughter, misconduct in public office, and/or offences under the Health and Safety at Work Act 1974.
"We will also be considering whether any potential disciplinary offences have been committed."
Members of the public raised the alarm when they became worried about Mr Briggs's behaviour in Marsh Road, Luton, and he was taken into custody.
Ms Cuneen said: "IPCC investigators have obtained and reviewed CCTV from the custody suite at Luton police station and at the junction of Marsh Road and Willow Way.
"A number of independent witnesses have been identified, accounts have been taken and IPCC investigators will continue to interview witnesses over the coming days. Local businesses have also provided CCTV footage to assist our investigation.
"But we still believe there are a number of other people who may have seen Leon on Monday and we are keen to speak to them, particularly if you were in the Marsh Road area or outside Luton police station."
Anyone with information is asked to contact the IPCC investigation team.
Mr Briggs' family issued a statement saying: "Leon was a loving father, son and brother. He was a kind, loyal, intelligent, caring person who put his family and others first.
"It is important to us to see the seriousness with which the IPCC are approaching the investigation into Leon's death."
A criminal investigation has started into the death of a man in police custody who had been detained under the Mental Health Act.