`` my wife , '' he said simply , and it occurred to grey , with a pang , that it had not been inadvertence or lack of caution that had impelled jamie fraser to come to his house , even with soldiers on his heels .