In olive trees, fluctuations in the onset of phenological stages have been reported due to weather conditions. The present study analyses the reproductive phenology of 17 olive cultivars grown in Elvas (Portugal) in 3 consecutive years (2012–2014). Through 2017–2022, the phenological observations continued with four cultivars. The phenological observations followed the BBCH scale. Over the course of the observations, the bud burst (stage 51) occurred gradually later; a few cultivars did not follow this trend in 2013. The flower cluster totally expanded phase (stage 55) was achieved gradually earlier, and the period between stages 51–55 was shortened, especially in 2014. Date of bud burst showed a negative correlation with minimum temperature (Tmin) of November–December, and, in ‘Arbequina’ and ‘Cobrançosa’, the interval stage 51–55 showed a negative correlation with both the Tmin of February and the Tmax of April, whereas in ‘Galega Vulgar’ and ‘Picual’ there was instead a positive correlation with the Tmin of March. These two seemed to be more responsive to early warm weather, whereas ‘Arbequina’ and ‘Cobrançosa’ were less sensitive. This investigation revealed that olive cultivars behaved differently under the same environmental conditions and, in some genotypes, the ecodormancy release may be linked to endogenous factors in a stronger way.