US R and B singer Jesse Braham, whose stage name is Jesse Graham, says the words from her 2014 hit Shake It Off  come from a song he wrote called "Haters Gone Hate".
He's started a legal case against her and is asking for Â£27 million pounds in compensation.
He also wants his name added as a writer on the track.
There's been no official comment from Taylor Swift or her representatives.
Shake It Off topped music charts around the world and reached number two in the UK.
The video for the song has been watched more than 1.1 billion times on YouTube.
Jesse Braham claims he has copyright ownership of the phrases "haters gone hate" and "playas gone play", which appear in the chorus of Taylor Swift's song.
He says he's spoken to Swift's record label, Big Machine, four or five times about the issue.
He originally asked to be named as a writer and asked for a selfie with Taylor Swift but he says he was told that wasn't possible.
Apart from the words being similar, the songs aren't the same musically.
American pop star Taylor Swift is being challenged over the lyrics in one of her most famous songs.