During a long war (1046–1056) as an ally of Duke Godfrey the Bearded of Lorraine against Emperor Henry III, Baldwin initially lost Valenciennes to Count Herman of Mons. However, when the latter died in 1049, Baldwin had his son, Baldwin VI, marry Herman's widow Richilde, and arranged that the children of her first marriage were disinherited, thus "de facto" uniting the County of Hainaut with Flanders. Upon the death of Henry III this marriage was acknowledged by treaty by Agnes of Poitou, mother and regent of Henry IV.