The man, who cannot be named, told Merthyr Crown Court the incidents took place at the Trealaw Clinic in Pentre, Rhondda Cynon Taff, in 2011.
Dr Brian Harris, 77, worked for the NHS and in a private practice in Cardiff for 40 years.
He denies 13 counts of sexual assault, indecent assault and rape against five men between 1991 and 2011.
The jury of seven women and five men was shown a police interview with one of his alleged victims.
Breaking down in tears, the 49-year-old support worker said he felt he could not talk about the alleged assaults at the time.
He said: "Who would listen? I hadn't been well - the biggest thing was, would I be believed?"
The man was diagnosed with bi-polar disorder and also suffered from depression.
He said Dr Harris quizzed him about his physical and sexual relationship with a man he was in a relationship with.
On one occasion, which he described as "bizarre", he said Dr Harris took him into a room they did not normally use and showed him how to do press-ups.
Another time, he said the doctor instructed him to undress so he could examine him and then touched him.
"I was very uneasy," he said. "I remember thinking, 'please don't touch me'."
Eventually he reported the behaviour to another member of the mental health team.
The trial continues.
A man has told a court how he was sexually assaulted by his psychiatrist during consultations.