During the 15th and 16th centuries, Cartaya became part of the dominions of the Zuñiga's House. The area was unsafe because it was frequently attacked by Berbers and Normans pirates. Until Marquess of Gibraleón, Pedro de Zuñiga and Manrique de Lara, didn't win a lawsuit with Alfonso de Guzmán, M. of Ayamonte and Lepe, for which a step could established by boat across the river Piedras and found next to him the population to be able to defend such a step, there was no certain normality in the zone. Seven years later, he gave the orders to initiate the works of the castle, located today to the west of the current urban area and of that only the court of weapon remains. The construction of the fortress, ideal refuge for the population, ends for consolidating the current settlement. Also on that date, the first church was built in Mudéjar style, on which subsequently got up the present Parish of San Pedro. His descendant, Álvaro of Zúñiga consolidated the familiar power in the Lower Andalusia and his eldest son, on having married the daughter of Juan Alfonso of Guzman III was done finally in dowry of Ayamonte, the nearby Lepe and La Redondela.