Crosby (born Henry Sturgis Crosby — his parents Stephen Van Rensslaer Crosby and Henrietta Marion Grew later changed his middle name to Grew) was born in Boston's exclusive Back Bay neighborhood. He was the product of generations of blue-blood English and Dutch American families, descended from the Van Rensselaers, Schuylers, Morgans, and Grews. His uncle was J. Pierpont Morgan Jr., one of the richest men in America at that time. His father's mother was the great-granddaughter of Peggy Schuyler, sister-in-law of Alexander Hamilton. Also among Harry's ancestors were Revolutionary War General Philip Schuyler, and William Floyd, a signer of the Declaration of Independence.