I am a honor student and ready to start my phd at top 3 ivys, so this is not about grading. 

Honestly, it is not about difficulty either, it is about his attitude towards students. He might be very smart, but he is kind of condescending in his teaching.  

Good luck with his next position, very much good luck with his next batch of students. 