The prince spent most of his time at Ludlow until he was twelve, when his father the king suddenly died. The prince had to go straight to London to take the throne, and his uncle, Earl Rivers, went with him. On the way, they were met by another of his uncles, Richard, Duke of Gloucester. The Duke was a younger brother of King Edward IV, and the king had given him the title of "Lord Protector" in the hope that he would look after Prince Edward until he was old enough to be the real king.