Most likely Carlo was born at Venosa, then part of the Kingdom of Naples, but little else is known about his early life. "His mother died when he was only seven, and at the request of his uncle, Carlo Borromeo for whom he was named, he was sent to Rome to be set on the path of an ecclesiastical career. There he was placed under the protection of his uncle, Alfonso (d. 1603), then dean of the College of Cardinals, later unsuccessful pretender to the papacy, and ultimately Archbishop of Naples." His brother Luigi was to become the next Prince of Venosa, but after his untimely death in 1584, Carlo became the designated successor. Abandoning the prospect of an ecclesiastical career, he married in 1586 his first cousin, Donna Maria d'Avalos, the daughter of Carlo d'Avalos, prince of Montesarchio and Sveva Gesualdo, princess of Venosa. They had a son, Emanuele.