sherry had undoubtedly retired hours ago , he decided , growing thoroughly annoyed with himself for rushing home from his mistress 's inviting arms to devote himself to her like some overardent nursemaid .