He was found unconscious and not breathing at Littlecroft in Theobald Street, Radlett, Hertfordshire, just after 00:30 GMT on 28 March and died later in hospital.
Police said officers were called to a "social gathering" and the death of a 35-year-old man was being probed.
Richard Stanley from Radlett Adult Parties said guests administered CPR until an ambulance arrived.
The man was taken to Watford General Hospital where he died shortly afterwards.
Hertfordshire Police said its investigation was ongoing but the death was being treated as unexplained pending the outcome of a post-mortem examination.
Mr Stanley, 66, who hosts parties with his partner, Janet Harrison, 58, said about 30 guests attended Friday's event.
He said the 15m covered pool is 2.5m deep and for "safety reasons diving is disallowed and not possible".
"I got a shout that someone had sunk in the pool so ran into the area, dived straight in fully clothed and pulled a Nigerian man to the pool edge," he said.
Mr Stanley said he was "one of the first to administer mouth to mouth breathing assistance" but there was also a qualified nurse and "other competent people" present.
"A lady from the 999 service gave me clear instructions which I shouted to the people administering CPR," he said.
"We continued doing that until the ambulance arrived a few minutes later."
Mr Stanley said he did not know what had caused the man to sink but confirmed the victim had been drinking.
"The adult aspect to the party added nothing to the risk of this happening," he said.
"Our thoughts are with [the man's] friends and relatives.... they will be welcome to come to see us and where his life so sadly and suddenly ended."
A man has died after being pulled from a swimming pool at a swingers' party.