Peabody was born in 1795 in what was then South Danvers (now Peabody), Massachusetts. His family had Puritan ancestors in the state. As one of seven children in a poor family, George suffered some deprivations during his childhood, and was able to attend school for only a few years. When he was a teenager, his father died, and he worked in his brother's shop to support his widowed mother and six siblings. He later expressed "I have never forgotten and never can forget the great privations of my early years". These factors influenced his later devotion to both thrift and philanthropy.