Media playback is not supported on this device
The 24-year-old joined the Millers in the summer of 2012 and scored 32 goals in 91 appearances.
It was reported last week that Agard had met with Leeds over a potential move but he was unable to agree a deal.
"We're pleased we've got him and we're pleased with the price we've got him for," Bristol City manager Steve Cotterill told BBC Points West.
"Even more importantly he turned down a bigger contract last week to come and sign for Bristol City. That's exactly what I wanted him to do.
"His goal ratio last year was excellent. I don't think you can ever have enough goal-scoring centre-forwards."
Agard will be available for Saturday's League One game with Rochdale.
The forward started his career as a trainee at Everton and had loan spells at Peterborough and Kilmarnock before joining Yeovil on a permanent deal in June 2011.
He made 34 appearances for the Glovers, scoring six goals, and joined Rotherham after only one season at Huish Park.
Agard's arrival at Ashton Gate means Cotterill now has a plethora of forwards to choose from, including last season's top-scorer Sam Baldock.
The City captain has been the subject of bids from Brighton but Cotterill insists: "We haven't bought Kieran Agard so that we can sell Sam Baldock.
"Anybody that's come in for Sam is nowhere near [the valuation] but I've got a sneaky feeling there's going to be a twist in this tale. I still think there might be someone else who's interested.
"I'd love for the transfer window to shut and for us to still have Sam. I can't promise that but all I can promise is if Sam ends up going somewhere, which we don't want to happen, it will be for a fee that measures up to his ability."
Bristol City have signed striker Kieran Agard from Rotherham for an undisclosed fee on a three-year deal.