The man got into difficulties at Portmellon near Mevagissey while walking with his pets at around 11:00 GMT, said Devon and Cornwall Police.
Officers called the death a "tragic accident".
Another member of his family tried in vain to help and the man was airlifted from the sea by a coastguard helicopter crew.
The family member, believed to be the man's stepdaughter, was rescued from the water by a member of the public.
A coastguard spokesman said: "Two people were recovered from the sea at Port Mellon near Mevagissey this morning.
"One was recovered from the sea by a member of the public and brought to shore.
"A second person was recovered from the sea by the Coastguard helicopter and was taken to hospital.
"The RNLI Lifeboat from Fowey and Mevagissey Coastguard Rescue Team were also in attendance."
It is not clear what has happened to the dog.
The airlift of the man was the first rescue operation by private firm Bristow which took over air and sea rescue services in Cornwall from RNAS Culdrose on Friday.
A holidaymaker has drowned after trying to rescue one of his dogs from the sea in Cornwall.