Elbert was given considerable freedom while being held prisoner, which was unusual given the typically harsh treatment of prisoners at the time. It is a family tradition, however, that this freedom exposed him to a plot upon his life. It was attributed to a gang of Tories who had every reason to dislike Elbert since he had been very active against them. Their plan was to have him killed by Indians. While strolling in the woods one day, Elbert encountered two Indians with guns aimed directly at him. He had always extended great kindness to the Indians whenever he had had dealings with them in the past. He made a secret signal to them, and they recognized him as a friend.