he pushed down a nagging irritation that not only had the woman recognized him and his drink , even though he could swear hed never seen her in his life , shed also easily assumed he would get falling-down drunk .