A search involving lifeboat crews, the coastguard and police was launched on Saturday following reports of a man clinging to rocks at St David's Head.
It ended on Sunday evening and there are no plans to resume unless further information is uncovered.
A coat, believed to belong to the man, was found containing money but there was no form of identification.
Dyfed-Powys Police said it was "urgently trying to identify the man, to build a picture of his activity leading up to Saturday and to make contact with his family".
He has been described as white European, aged in his 60s, of lean build, with a long face, short hair and distinctive white-coloured eyebrows.
He was wearing a dark coloured day rucksack, dark blue/black trousers and a dark blue long sleeved shirt or jumper.
Police said, based on his accent, he was not thought to be from the UK.
Police have asked for help to identify a man thought to be missing in the sea off Pembrokeshire.