Carney died at the Boston City Hospital on December 9, 1908, of complications from an elevator accident at the Massachusetts State House, where he worked for the Department of State. His body lay in repose for one day at the undertaking rooms of Walden Banks, 142 Lenox Street, at the wish of his wife and daughter. He was buried in the family plot at Oak Grove Cemetery in New Bedford, Massachusetts. Engraved on his tombstone is an image of the Medal of Honor.