Copthall House, at the railway station entrance at junction six of the ring road, is being cleared so new Coventry City Council offices can be built.
Lanes on the ring road and Manor Road and  will be closed until Monday. Work on the site is expected to continue into 2015.
A large section of the five-storey block is due to be removed on Sunday.
Work is scheduled for between 05:00 and 09:30 BST. The council said it had planned for the most disruptive work around the station to take place at off-peak times.
Ian Stringer, from site property agents GVA , said the buildings being demolished were "not particularly good brutalist examples".
Councillors will continue to be based at the Grade II-listed council house but will move some departments and administration to Friargate as part of plans to move from 27 council buildings to nine.
Public council services will move to a new "one-stop centre" in Broadgate.
Kevin Maton, cabinet member for business, enterprise and employment said: "The start of demolition on site is an exciting point in the Friargate project. We are witnessing a period of massive change in Coventry."
He said the development had potential to create up to 15,000 jobs.
Twenty-seven new buildings are planned for the site, including offices, hotels, houses and shops, and a new bridge will link the city centre and railway station.
Property agent Mr Stringer said: "Nothing of this style, scale and quality has happened in Coventry for three or four decades. Businesses will want to be there."
A prominent 1970s office block in Coventry is being demolished as part of the city's £59m Friargate development.