Anthony Pascoe, 56, of Gwithian Towans, Hayle, Cornwall, admitted sexual activity with the woman who has learning difficulties.
The judge at Truro Crown Court jailed Pascoe for four years and three months for the "appalling" offences.
Pascoe sent the woman "sexually explicit" text messages and had sex with her at the care facility.
Pascoe also had sex with the victim at her home, after which she became very upset and told her parents who informed the police.
Judge Simon Carr said Pascoe was fully aware of his victim's learning disabilities.
He said: "What is clear to me is that those difficulties were such that she was particularly vulnerable and susceptible to your approach."
The court heard the victim could not use a mobile phone or go shopping without assistance.
The judge said Pascoe's actions represented "a classic case of the grooming of the vulnerable" and had a "catastrophic" impact on the victim.
"You took away what little independence she had," he said.
Pascoe admitted two offences of possession of extreme pornography and three of sexual activity with a person with a mental disorder by a carer.
Following the hearing, the victim's father told the BBC it was "horrifying" to see his daughter abused in what he believed was a protected environment.
"We were trying to prepare her for an independent life and now she is terrified in case anything happens to us," he said.
A care worker has been jailed for "grooming" and having sex with a vulnerable woman in his care.