Vanderbilt was born on February 20, 1924, in Manhattan, New York City, the only child of railroad heir Reginald Claypoole Vanderbilt of the Vanderbilt family and his second wife, Gloria Morgan Vanderbilt. When Vanderbilt was born, her father was heard to exclaim in delight, "It is fantastic how Vanderbilt she looks! See the corners of her eyes, how they turn up?" She was baptized in the Episcopal church by Bishop Herbert Shipman as Gloria Laura Vanderbilt. After her father's death, she was confirmed and raised in the Catholic Church, to which her mother belonged. From her father's first marriage to Cathleen Neilson, she had one elder half-sister, Cathleen Vanderbilt.