Following the 2002 season, on February 27, 2003, Harrison was released by the Chargers. Two weeks later, on March 13, Harrison landed with the Patriots, signing a six-year deal. At the time, the Patriots had both Harrison and fellow Pro Bowl safety Lawyer Milloy under contract. Throughout the offseason, though, the Patriots and Milloy were involved in contract negotiations, with the Patriots requesting Milloy take a pay cut or be released. Milloy did not comply, and on September 2, he was released.