Former Yugoslavia defender Mihajlovic, 46, has signed a two-year deal after leaving Sampdoria earlier this month.
Inzaghi, 41, who played for Milan from 2001 until his retirement in 2012, replaced former team-mate Clarence Seedorf as Milan coach last June.
Milan finished in 10th place in Serie A, 35 points behind champions Juventus, after only 13 wins in 2014-15.
Mihajlovic played for Roma, Sampdoria, Lazio and Inter Milan during a 20-year senior career in which he won 63 international caps.
Having retired while at Inter, he began his coaching career there as assistant to Roberto Mancini.
He had brief spells in charge at Bologna, Catania and Fiorentina before he was appointed coach of Serbia in April 2012, winning seven and losing eight of his 19 matches in charge.
Mihajlovic joined Sampdoria in November 2013 and took them to 12th place that season, improving to seventh place last term.
Inzaghi, meanwhile, became Milan youth-team coach after retiring as a player before succeeding Seedorf, who only lasted five months in the role.
AC Milan have won 18 Serie A titles and seven European Cups.
AC Milan have appointed former Serbia boss Sinisa Mihajlovic as their new coach after sacking Filippo Inzaghi.