The 31-year-old caused the defending champion all sorts of trouble throughout the fight in Germany.
Smith, the British champion, worked hard throughout, setting the tempo from the start.
But Abraham put in a strong last round and two judges gave it 117-111 and the third 119-109.
Afterwards Smith described the scoring as 'disgusting'.
"That's twice I have been on the end of a wrong decision and it has shattered my world," he told Sky Sports.
"I knew I had the beating of him but I did a lot right and was outworking him in every round.
"I believed I had done enough and won the fight but the judges didn't. I feel robbed."
The Liverpudlian, who was stopped by James DeGale and George Groves in 2010 and 2011 respectively, needed to make a solid start and he started at a fast pace, putting in a good effort in the opening couple of rounds but without seriously troubling his opponent.
Abraham gradually came more into the fight and, with some solid defence, he started to pile on the pressure by the midway point of the fight.
Smith tried to force the issue in the seventh round and began to land the bigger punches and, with Abraham showing signs of vulnerability, he was rocked by a left hook from Smith in the ninth round.
Abraham regrouped and started to look more dangerous by the 10th round but he was also looking more tired and Smith dug deep for one last effort but it was not enough.
Liverpool boxer Paul Smith put in the performance of his career but lost out on points to Germany's Arthur Abraham in their WBO super-middleweight bout.