After spending an extended time traveling through Europe and then establishing himself as a successful merchant in Boston, Lowell married his first wife, Susan Cabot Lowell (1801–1827), a daughter of his uncle, Francis Cabot Lowell. Together, they would have two children, Susan Cabot and John. Lowell's wife died during childbirth in 1827. Their son, John, would be appointed to the U.S. District Court in 1865 by President Abraham Lincoln, and in 1878, appointed to the U.S. Circuit Court by President Rutherford B. Hayes. John Amory's grandson, James Arnold Lowell, would also go on to become a Federal Judge. Lowell's wife, Susan Cabot, who was a great-granddaughter of Edward and Dorthy (Quincy) Jackson, would connect their children and their descendants to those of the Holmeses of Boston, a family that includes poet Dr. Oliver Wendell Holmes, Sr., and U.S. Supreme Court justice and Civil War hero, Oliver Wendell Holmes, Jr.