To come to the point, can we have a word with you in private? asked Sir James.
Sir James asked of the person to have a word with him in private.
Elizabeth asked of the person to have a word with her in private.