Endurance performance is positively associated with hematological adaptations; therefore, high total hemoglobin mass and intravascular volumes are commonly observed in high-level endurance athletes. However, it is still unclear whether the fluctuations in exercise capacity that typically occur in endurance athletes during the annual training cycle are directly associated with changes in hematological adaptations, which appear to be relatively stable during this time. To better understand this issue, a study was conducted with 10 Olympic rowers, who followed the same training program. Athletes underwent laboratory testing in the competitive and the general preparation phase of an annual training cycle (a 34% reduction in training volume). This included a graded exercise test on a rowing ergometer (GXT) and blood measurements of hemoglobin concentration (Hb), total hemoglobin mass (tHb-mass), plasma volume (PV), and blood volume (BV). Decreases in maximal values of power relative to body mass (p = 0.028), lactate concentration (p = 0.005), and heart rate (p = 0.017) in the GXT were registered. At the same time, absolute (p = 0.017) and relative (p = 0.005) PV decreased. Changes in PV (rS = 0.842, p = 0.002) and BV (rS = 0.818, p = 0.004), but not in tHb-mass (rS = 0.588, p = 0.074) and Hb (rS = −0.188, p = 0.602), were significantly correlated with changes in maximal power in the GXT. Our results indicate a close relationship between changes in intravascular volumes and maximal exercise capacity after a period of reduced training loads in elite endurance athletes.