The 24-year-old centre-back has agreed a four-year contract with the Seagulls.
Duffy, who turned down a new deal at Ewood Park earlier this month, scored seven goals in 69 appearances during two years with Rovers.
"I'm delighted we've completed the signing of Shane, he's an excellent addition for us," Brighton manager Chris Hughton said.
Republic of Ireland international Duffy scored two own goals and was sent off in Blackburn's 2-1 defeat at Cardiff on 17 August, but he returned from suspension to net the winner in the EFL Cup second-round tie against Crewe on Tuesday.
The defender, who featured twice for his country at Euro 2016, is Brighton's seventh signing of the summer transfer window.
Find all the latest football transfers on our dedicated page.
Brighton have signed Shane Duffy from Championship rivals Blackburn Rovers for an undisclosed fee.