The 32-year-old collided with world champion Peter Sagan before hitting the barriers in a sprint finish on stage four in Vittel on Tuesday.
Sagan left the Tour after being disqualified for causing the crash which ended Cavendish's race.
"Please know that it's sport and I have a family," Cavendish said on Twitter.
"Everyone is entitled to their opinions. But vile and threatening comments on social media to myself and my family aren't deserved."
Speaking in a video posted on Wednesday, the Isle of Man rider continued: "I'm paying now as a 32-year-old for the attitude I had as a kid. Unfortunately this will never leave me and I'll always deal with the brunt of people's personal opinions.
"But I'd ask you all to that respect that and please not send threatening or abusive language to myself and my family.
"Thank you to everyone for your well wishes and I hope you continue to enjoy the rest of the Tour de France."
Media playback is not supported on this device
Cavendish said Slovak Sagan, 27, elbowed him during the sprint finish to stage four but later said he had "no hard feelings" towards the rider and that they remain friends.
Sagan's Bora-Hansgrohe team protested against his disqualification, but race organisers would not allow an appeal.
The Tour continued on Wednesday as Britain's Chris Froome took the yellow-jersey after finishing second behind Italy's Fabio Aru on stage five.
Froome, 32, now leads Welshman and team-mate Geraint Thomas by 12 seconds in the general classification.
Mark Cavendish has appealed for an end to "vile and threatening" social media comments to him and to his family after his Tour de France exit.