The free pennies will be marked with their year of birth.
The Duchess of Cambridge gave birth to a girl on Saturday.
A spokesman for the Llantrisant-based Royal Mint said the gesture was in keeping with "an age-old tradition to mark a new birth with a gift of silver for good luck".
Parents of newborns who share a birthday with the new princess will have to register the birth of their child on the Royal Mint's Facebook page within 60 days to receive the commemorative gift.
The first family to receive one of the silver pennies was Chantal and Miguel Abel, and their daughter Sofia who arrived at the Chelsea & Westminster Hospital in London at 11:30 BST on Saturday.
They said in a statement: "It's a special day for us and we're excited to be one of the first families to receive a lucky silver penny and be part of the celebrations."
The palace said the Duke of Cambridge was present for the birth of the new princess, who weighs 8lbs 3oz.
Both Catherine and her daughter are "doing well", it added.
Shane Bissett, director of commemorative coin at the Royal Mint said: "The birth of the new royal baby is a joyous occasion not just for their Royal Highnesses the Duke and Duchess of Cambridge but also for the whole nation, as we celebrate another remarkable milestone in their life journey together.
"However, it is also a special day for many mothers and fathers across the country as they too welcome the arrival of their new babies, hence why we wanted to extend this historical moment to them with a unique and timeless gift that can be treasured for years to come."
The Royal Mint gave away 2,013 silver pennies to babies born on the same day as the royal couple's first child Prince George.
To mark the birth, a £5 coin will also be released by the Mint - as it did for the birth of the prince.
The Royal Mint is to gift 2,015 babies born on the same day as the new Royal baby with a "lucky" silver penny.