On balance I didn't like him at all, but the social conservatism and twisting of statute law weren't what put sand in my gears.

Omnibus budget bills so he could pass legislation without debate.  Callous disregard for journalists, so the only challenges were the puppet show that is Question Period.  Autocratic to the extreme in his own party.  Bumping up the PMO and staffers by 21%.  More or less declaring war on the Public Service, instead of working with them to improve things.  The debacles with Phoenix and Shared Services Canada, because he insisted on booking savings long before those cakes were fully baked.  Boasting about his fiscal aptitude while running deficits 8 years out of 10 (and cutting services).

And finally, and I know this one is petty, but the unshakeable and creepy feeling that he was a male Stepford Wife.  He had about as much emotion as a Sears mannequin, and a man who never shows his feelings is a man I simply cannot trust.