It was at Argentan that the friendship between Carrouges and Le Gris began to deteriorate, as Le Gris rapidly became a favourite of Count Pierre. While Carrouges was overlooked, Le Gris was rewarded for service to the Count, inheriting his father's lordship of the castle at Exmes and being granted a newly purchased estate at Aunou-le-Faucon. Carrouges became jealous of his friend and the two men soon became rivals at the court. A year after entering Count Pierre's service, tragedy struck Carrouges as both his wife and son died of unknown but natural causes. In response, Carrouges left home and joined the service of Jean de Vienne accompanied by a retinue of nine squires. With this force, under the overall command of King Charles V, Carrouges distinguished himself in minor actions against the English in Beuzeville, Carentan and Coutances in a five-month campaign, during which over half his retinue were killed in battle or by disease.