During the decline of the Roman Empire in the 4th century, the villa gradually fell into disuse and was partially ruined as valuable statues and marble were taken away. The facility was used as a warehouse by both sides during the destructive Gothic War (535–554) between the Ostrogoths and Byzantines. Remains of lime kilns have been found, where marble from the complex was burned to extract lime for building material. Building material was also reused by the Christians to build basilicas and other buildings.