In 1936 he married his second wife Phyllis, who was Mary Kate's sister. Another sister was married to Fine Gael leader General Richard Mulcahy, and a brother was Fianna Fáil minister, James Ryan. O'Kelly died in Dublin.