Agnes was born in 1268 into the noble Segni family in Gracciano, a frazione of Montepulciano, then part of the Papal States. At the age of nine, she convinced her parents to allow her to enter a Franciscan monastery of women in the city known as the "Sisters of the Sack", after the rough religious habit they wore. They lived a simple, contemplative life. She received the permission of the pope to be accepted into this life at such a young age, which was normally against Church law.