A 14-year-old girl managed to climb out before the car slipped from the promenade in Burntisland, while three others, aged 10, 13 and 14, managed to get ashore before the RNLI arrived.
Two members of the public helped free the man, who was wearing a leg brace which had become jammed.
He was taken ashore by lifeguards.
The incident happened in front of the Beacon Leisure Centre at about 15:40. All five casualties were treated for shock and one was given oxygen.
Sisters Kayleigh Reid, 22, and Vicki Reid, 19, who were on lifeguard duty in Burntisland at the time, heard a revving engine followed by a splash.
Vicki Reid paddled to the car on a rescue board with her sister arriving moments later having swum 250 metres. She had radioed the coastguard at the same time.
Emergency services arrived at the scene quickly with the Queensferry RNLI lifeboat sent to assist.
The man was taken to shore on the rescue board.
Kayleigh Reid said: "We heard a car revving and we turned around to see it fall into the water and a lot of people started panicking.
"The car had fallen about two metres and was in water that was about three to four metres deep.
"We are always ready to do rescues in the water, we were not expecting a car, normally its just a swimmer, but we are trained on how to get people out of the water.
"Everyone involved was very, very lucky. It could have ended up a lot worse, but everyone reacted so quickly and all the emergency services got on the scene so fast that everyone has managed to walk away."
RNLI lifeguard manager Tim Doran said: "This is only the second year that we've had RNLI lifeguards on beaches in Fife.
"I'm really pleased with the way our lifeguards and members of the public reacted to ensure the safety of everyone involved in this incident."
A 56-year-old man and four girls have escaped injury after their car fell about 2m (7ft) into the water from a promenade in Fife.