From 1331 the feudal lords began to recognize the rights of James, and in 1333 the recognition was total. Then John transferred his rights to his sister-in-law, Catherine of Valois, titular Empress of Constantinople, wife of Philip I of Taranto, whose stepson Robert claimed her rights until 1346 when she died. Then the claim was issued by the son of Philip and Catherine, Philip II of Taranto. In 1349 James was succeeded by his son James IV (II of the Morea). In 1364 Robert of Taranto, stepson of Catherine and eldest surviving son of Philip I of Taranto, died. In 1373 Philip II transferred his rights to his cousin, overlord and former sister-in-law Queen Joan I of Naples, whose third husband James IV of Majorca, when he died in 1375, left her his own claim to the principality, at which point she became more or less uncontested Princess of Achaea. However, when Joan was imprisoned in Naples in 1381, another, much younger, James, James of Baux, grandson of Catherine and nephew of Philip II, who in 1374 had become titular Emperor of Constantinople, used the opportunity and seized Achaea. In 1383, Achaea was annexed by Charles III of Naples, successor and murderer of Queen Joan of Naples, who was the grandson of John of Durazzo, and James of Baux was driven away. In 1383 the Vicary government began, lasting until 1396, under the Durazzo kings of Naples.