Sajmon Brzezinski, 43, took the man's bank card, only giving him £20 to £30 of his £480 wages, Nottingham Crown Court heard.
The landlord sent him to work at Sports Direct in Shirebrook, Derbyshire, having brought him to the country from Poland, and later to a farm.
Brzezinski was arrested after the man told the farm about his circumstances.
Nottingham Crown Court heard Brzezinski, of Overdale Road, Old Basford, Nottingham, brought the man from Poland to the city in 2011.
His tenant worked at Sports Direct from then until 2013 when he went to work at a farm in Arnold, Nottinghamshire.
Brzezinski, who owned multiple cars and properties, including a "palace" in Poland, was arrested after the worker told his line manager about the situation at the farm's Christmas party.
Bosses at the farm reported it, which led to an investigation by Nottinghamshire Police and the Gangmasters and Labour Abuse Authority (GLAA).
The court also heard that Brzezinski had tricked another vulnerable man into handing over most of the money he earned - he worked at Sports Direct and later the farm too.
The two victims were terrified of Brzezinski, the court heard.
One victim could not sleep and had nightmares about being killed. When investigators asked to speak to him, he started shaking and crying uncontrollably, and had to go to hospital with chest pains.
Sentencing, Judge Stuart Rafferty QC said: "Each of these men placed his trust in you. And you repaid that with cynicism and greed."
Sports Direct has not yet responded to requests for a comment.
Brzezinski admitted one charge of human trafficking, two of forced labour and one of fraud at a previous hearing.
A landlord who forced his tenant into work to fund his lavish lifestyle has been jailed for eight years.