Fisher was five feet seven inches tall and stocky with a round face. In later years, some suggested that Fisher, born in Ceylon of British parents, had Asian ancestry due to his features and the yellow cast of his skin. However, his colour resulted from dysentery and malaria in middle life, which nearly caused his death. He had a fixed and compelling gaze when addressing someone, which gave little clue to his feelings. Fisher was energetic, ambitious, enthusiastic and clever. A shipmate described him as "easily the most interesting midshipman I ever met". When addressing someone he could become carried away with the point he was seeking to make, and on one occasion, the King asked him to stop shaking his fist in his face. He was considered a "man who demanded to be heard, and one who didn't suffer fools lightly".