Whitfield started for the New York Giants at left tackle during the 2006 season filling in after starter Luke Petitgout went down with injury. His time as a starter was marked by two episodes of him losing his cool and drawing costly personal fouls, twice on headbutts in key spots against the Dallas Cowboys and later in a week 16 loss against the New Orleans Saints. These actions lead to him receiving the nickname "Head-butt Bob" from many Giant fans as well as local columnists. Due to these actions and general inconsistent play, Whitfield was benched for the final game of the season against the Washington Redskins and the subsequent playoff game against the Philadelphia Eagles.