Citation: Manresa-Rocamora, A.;
Fuertes-Kenneally, L.; Blasco-Peris,
C.; Sempere-Ruiz, N.; Sarabia, J.M.;
Climent-Paya, V. Is the Veriﬁcation
Phase a Suitable Criterion for the
Determination of Maximum Oxygen
Uptake in Patients with Heart Failure
and Reduced Ejection Fraction? A
Validation Study. Int. J. Environ. Res.
Public Health 2023, 20, 2764. https://
doi.org/10.3390/ijerph20042764
Academic Editor: Cristian Álvarez
Received: 12 January 2023
Revised: 31 January 2023
Accepted: 2 February 2023
Published: 4 February 2023
Copyright:
© 2023 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed
under
the
terms
and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).
International  Journal  of
Environmental Research
and Public Health
Article
Is the Veriﬁcation Phase a Suitable Criterion for the
Determination of Maximum Oxygen Uptake in Patients with
Heart Failure and Reduced Ejection Fraction?
A Validation Study
Agustín Manresa-Rocamora 1,2
, Laura Fuertes-Kenneally 1,3
, Carles Blasco-Peris 1,4
,
Noemí Sempere-Ruiz 1,2
, José Manuel Sarabia 1,2,*
and Vicente Climent-Paya 1,3
1
Institute for Health and Biomedical Research of Alicante (ISABIAL), 03010 Alicante, Spain
2
Department of Sport Sciences, Sports Research Centre, Miguel Hernández University of Elche,
03202 Elche, Spain
3
Cardiology Department, Dr. Balmis General University Hospital, 03010 Alicante, Spain
4
Department of Physical Education and Sport, University of Valencia, 46010 Valencia, Spain
*
Correspondence: jsarabia@umh.es; Tel.: +34-96-522-25-68
Abstract: The veriﬁcation phase (VP) has been proposed as an alternative to the traditional criteria
used for the determination of the maximum oxygen uptake (VO2 max) in several populations.
Nonetheless, its validity in patients with heart failure with reduced ejection fraction (HFrEF) remains
unclear. Therefore, the aim of this study was to analyse whether the VP is a safe and suitable
method to determine the VO2 max in patients with HFrEF. Adult male and female patients with
HFrEF performed a ramp-incremental phase (IP), followed by a submaximal constant VP (i.e., 95%
of the maximal workload during the IP) on a cycle ergometer. A 5-min active recovery period (i.e.,
10 W) was performed between the two exercise phases. Group (i.e., median values) and individual
comparisons were performed. VO2 max was conﬁrmed when there was a difference of ≤ 3% in peak
oxygen uptake (VO2 peak) values between the two exercise phases. Twenty-one patients (13 males)
were ﬁnally included. There were no adverse events during the VP. Group comparisons showed no
differences in the absolute and relative VO2 peak values between both exercise phases (p = 0.557
and p = 0.400, respectively). The results did not change when only male or female patients were
included. In contrast, individual comparisons showed that the VO2 max was conﬁrmed in 11 patients
(52.4%) and not conﬁrmed in 10 (47.6%). The submaximal VP is a safe and suitable method for the
determination of the VO2 max in patients with HFrEF. In addition, an individual approach should be
used because group comparisons could mask individual differences.
Keywords: cardiorespiratory ﬁtness; VO2 max; HFrEF; exercise testing; respiratory exchange ratio;
gradual exercise test; VO2 peak
1. Introduction
Heart failure with reduced ejection fraction (HFrEF) is a cardiovascular disorder
characterised by symptoms of breathlessness, ﬂuid retention, and exercise intolerance [1–3].
The maximal ramp or step incremental exercise test, coupled with breath-by-breath and
gas exchange measurements, is widely used in patients with HFrEF to measure maximum
oxygen uptake (VO2 max) and for risk stratiﬁcation [4–6]. VO2 max is deﬁned as the
physiological limit of oxygen utilisation [7] and is considered a strong predictor of mortality
in patients with HFrEF [8,9]. In fact, VO2 max is considered a prognostic factor in advanced
heart failure and is currently used as a key criterion for the selection of candidates for heart
transplantation (i.e., ≤14 mL·kg−1·min−1) [4].
The measurement of VO2 max requires the patient to perform a maximal exercise effort
(i.e., volitional exhaustion) and thus might be substantially underestimated due to muscle
Int. J. Environ. Res. Public Health 2023, 20, 2764. https://doi.org/10.3390/ijerph20042764
https://www.mdpi.com/journal/ijerph
Int. J. Environ. Res. Public Health 2023, 20, 2764
2 of 10
fatigue, breathlessness, and reduced motivation (i.e., submaximal exercise test). In these
circumstances, the peak oxygen uptake (VO2 peak) instead of the VO2 max is obtained.
Consequently, it is important to determine the criteria to accurately categorise an effort
as maximal. The primary criterion used to verify maximal exercise relies on the presence
of the oxygen uptake (VO2) plateau, which is deﬁned as no increase in VO2 despite an
increase in workload rate [10,11]. Nonetheless, data indicate that only a small percentage
of VO2 assessments actually exhibit a VO2 plateau [12–15], supporting the argument that
this physiological phenomenon is not necessary to acutely determine the VO2 max. Thus,
secondary criteria such as the value of the respiratory exchange ratio (RER)—which is
the most frequently used variable in cardiac patients—age-predicted maximal heart rate
(HR), or blood lactate concentrations are commonly used to verify that a maximal exercise
effort has been achieved [13,16,17]. However, evidence suggests that these criteria lack
validity since they can be met with either a maximal or submaximal exercise effort or even,
not reached at all despite a maximal effort [11,18,19]. In summary, traditional criteria (i.e.,
including both primary and secondary) are not reliable methods to ensure that the VO2
max is reached at the end of an incremental exercise test [20].
In an attempt to overcome the shortcomings of traditional criteria, a new criterion for
the determination of the VO2 max has emerged, known as the veriﬁcation phase (VP) [13].
The VP is a constant-load phase performed following the incremental phase (IP) and a
short recovery period (e.g., 3–5 min). Other protocols have also been used previously [20].
Regarding its intensity, it can be performed either above (i.e., supramaximal veriﬁcation
phase) or below (i.e., submaximal veriﬁcation phase) the peak work rate attained in the
previous IP [10,15]. There is evidence demonstrating that the VP is an adequate standard
for validating the VO2 max in healthy individuals [21] and patients with a wide range of
pathologies [22].
In this regard, Bowen et al. [23] investigated whether the submaximal VP was a valid
method to determine the VO2 max in patients with HFrEF. According to the authors, the
VP was well tolerated by patients with HFrEF, and its precision was greater than that of
secondary criteria (i.e., RER). Nonetheless, only male patients were included, and further
research is needed to determine whether VP is suitable and well tolerated by female patients
with HFrEF. Furthermore, although both group and individual comparisons can be used
to validate the VO2 max, group comparisons could mask individual differences between
the VO2 peak values attained in each exercise phase [19,20]. Also, the clinical utility of the
exercise test is its application to the individual rather than the group. For these reasons,
individual comparisons are more useful than group comparisons [24]. In order to perform
these individual comparisons and assess whether or not the VO2 max was reached, a
standard cut-off point should be established, preferentially using relative differences (e.g.,
≤3%) [20]. In contrast, Bowen et al. [23], who included group and individual comparisons,
carried out statistical comparisons. In this study, the VO2 max was conﬁrmed when
statistical signiﬁcance was not reached (p > 0.050). Nonetheless, the use of statistical
comparisons is a ﬂawed approach because it is designed to detect differences and depends
on the sample size [25]. Thus, the use of different approaches to conduct individual
comparisons and conﬁrm the VO2 max warrants future studies in patients with HFrEF.
Therefore, the main purpose of the current study was to investigate the utility of the
submaximal VP for validating the VO2 max in male and female patients with HFrEF. In
addition, we compared the level of agreement between the VP and traditional (i.e., RER)
criteria for the determination of the VO2 max. Based on previous evidence, we hypothesised
that the submaximal VP would be an adequate criterion to verify the VO2 max in male
and female patients with HFrEF when an individual approach is used, and no agreement
would be found between both criteria.
Int. J. Environ. Res. Public Health 2023, 20, 2764
3 of 10
2. Materials and Methods
2.1. Patients
Participants needed to fulﬁl the following inclusion criteria to be eligible: (a) male
or female aged between 50 and 70 years old; (b) diagnosed with HFrEF (left ventricular
ejection fraction < 50%); (c) stable phase of the disease with no recent hospitalisation or
visit to the emergency department due to heart failure (within the last six months before
the beginning of the study); (d) New York Heart Association (NYHA) functional class I, II,
or III; (e) under treatment with B-blockers; and (f) sedentary (i.e., not involved in exercise
training for six months). The exclusion criteria were: (a) use of intravenous diuretics in the
last six months; (b) unstable angina or evidence of severe ventricular arrhythmia; (c) atrial
ﬁbrillation; (d) supraventricular arrhythmias; (e) chronic obstructive pulmonary disease;
(f) recent of haemoglobin concentrations outside optimal parameters (13–16.5 g·dL−1);
(g) physical limitations that impeded the completion of the ergometry; and (h) the presence
of ischaemia, arrhythmias, or high frequency of ectopic heartbeats. All patients were fully
informed and signed the informed consent before any procedure related to the study was
performed. The protocol of this study was approved by the competent ethics committee of
the host institution (PI2021-177).
2.2. Measurements
Participants performed a symptom-limited exercise test which comprised two phases;
(a) the ramp-incremental exercise phase (i.e., IP); and (b) the steady-state exercise phase (i.e.,
VP). The test was carried out on an electromagnetically braked cycle ergometer (SanaBike
500 easy, Truchtelﬁnger, Germany). Before the start of the IP, a 3 min warm-up at 10 W and a
cadence of 50 revolutions per minute (rpm) was performed. The IP ended when the patient
reached volitional exhaustion or was unable to maintain a cadence of at least 45 rpm. The
exercise test was terminated, and the VP was not carried out in the presence of symptoms
of ischaemia or multifocal ectopic heartbeats (symptom-limited). Otherwise, a free-cadence
recovery period of 5 min at 10 W was performed after ﬁnishing the IP. Subsequently, the
VP was carried out at 95% of the maximum power reached during the IP. Throughout
the protocol, gas exchange was recorded with the Metalyzer 3B gas analyser (CORTEX
Biophysik, Leipzig, Germany), and HR was monitored with a 12-lead electrocardiograph.
Patients were asked to fast (at least three hours prior to the test), as well as to refrain from
strenuous physical activity (24 h), alcohol, and smoking (three hours prior).
2.3. Data and Statistical Analyses
Gas exchange and ventilatory variables were analysed to remove atypical breaths
(four standard deviations from the local mean) due to swallows, coughs, and so on [26].
VO2 peak was deﬁned as the highest VO2 occurring during each exercise phase (i.e., IP and
VP). VO2 peak, as well as the remaining ventilatory variables obtained at exercise peak (i.e.,
VCO2, oxygen pulse, RER, VE, VE/VCO2, and VE/VO2), were identiﬁed using a 12-breath
rolling average [23]. Breathing frequency and HR were averaged over 10 s.
Data are displayed as median (25th and 75th percentiles) and frequency (percentage)
for continuous and categorical variables, respectively, unless stated otherwise. Overall, the
Fisher-Pitman permutation test [27] and the non-parametric 95% conﬁdence interval (CI)
of the difference [28] were used to conduct between-phase comparisons (i.e., VP vs. IP).
The Bland-Altman plot was used to test the agreement between VO2 peak values measured
during the IP and VP.
Individual comparisons between VO2 peak values reached during the two exercise
phases were also conducted. In this regard, the IP-derived VO2 peak was conﬁrmed
(i.e., VO2 max) if the difference with the VP-derived VO2 peak value was ≤ 3%. After-
wards, patients were classiﬁed into two groups, depending on whether IP-derived VO2
max values were conﬁrmed or not. Fisher’s exact test, Mann-Whitney test, and Bonett-
Price 95% CI were used to conduct between-group comparisons (i.e., conﬁrmed vs. not
conﬁrmed groups).
Int. J. Environ. Res. Public Health 2023, 20, 2764
4 of 10
The traditional criterion (i.e., RER peak ≥ 1.10) was also used to verify VO2 max [17].
Compared to the VP, the results were classiﬁed as follows: (a) agreement, RER ≥ 1.10
and VO2 max conﬁrmed by VP or RER < 1.10 and VO2 max not conﬁrmed by VP; (b)
false positive, RER ≥ 1.10 and VO2 max not conﬁrmed by VP; and (c) false negative,
RER < 1.10 and VO2 max conﬁrmed by VP. The Kappa index was used to analyse the
degree of agreement between the two criteria (i.e., RER vs. VP). All tests were two-sided,
and p values ≤ 0.050 were considered signiﬁcant. All analyses were performed using
STATA software (version 16.0; Stata Corp LLC, College Station, TX, USA).
3. Results
3.1. Patients
Thirty patients with HFrEF (22 males; 73.3%) fulﬁlled the inclusion criteria to be
eligible to participate in the current study. Nonetheless, we excluded a total of eight patients
(26.6%) because the VP was considered contraindicated (i.e., symptoms of ischaemia,
arrhythmias, or high frequency of ectopic heartbeats during the IP). Moreover, one patient
(3.3%) did not complete the VP due to knee pain and was also excluded from the analysis.
All excluded patients were male. Therefore, 21 patients (13 males; 61.9%) were ﬁnally
included. The characteristics of these patients are shown in Table 1. No adverse events
occurred during the exercise tests. The median age was 64.0 years (57.5; 68.5), and the
median left ventricular ejection fraction was 39.1% (33.0; 43.1). Ischemic etiology was
the cause of HFrEF in almost half of the patients. Most of the included patients were
smokers (81%).
Table 1. Baseline participant characteristics.
Variable
n = 21
Conﬁrmed Group
(n = 11)
Not Conﬁrmed Group
(n = 10)
p
Age, years
64.0 (57.5; 68.5)
64.0 (56.0; 66.0)
65.0 (58.0; 69.3)
0.717
Height, cm
164 (158; 171)
164 (157; 168)
164 (159; 174)
0.617
Weight, kg
70.0 (66.5; 87.3)
72.0 (64.0; 88.5)
69.6 (67.8; 80.6)
0.850
Body mass index, kg/m2
27.8 (24.8; 31.6)
27.8 (25.1; 32.0)
27.7 (24.1; 31.4)
0.557
LVEF, %
39.1 (33.0; 43.1)
39.0 (33.0; 45.0)
37.5 (32.3; 43.0)
0.414
Male (%)
13 (61.9)
7 (63.6)
6 (60.0)
0.999
Ischemic etiology (%)
10 (47.7)
6 (54.6)
4 (40.0)
0.670
Diabetes mellitus (%)
9 (42.9)
4 (36.4)
5 (50.0)
0.670
Hypertension (%)
9 (42.9)
4 (36.4)
5 (50.0)
0.670
Dyslipidaemia (%)
11 (52.4)
6 (54.6)
5 (50.0)
0.999
Smokers (%)
17 (81.0)
11 (100)
6 (60.0)
0.035
ICD (%)
9 (42.9)
5 (45.5)
4 (40.0)
0.999
Drug therapy:
ACEI/ARBs (%)
8 (38.1)
4 (36.4)
4 (40.0)
0.999
ARNI (Sac/Val) (%)
13 (61.9)
7 (63.4)
6 (60.0)
0.999
MRA (%)
17 (81.0)
9 (81.2)
8 (80.0)
0.999
Antiplatelet (%)
8 (38.1)
4 (36.4)
4 (40.0)
0.999
Anticoagulants (%)
2 (9.5)
2 (18.2)
0 (0)
0.476
Diuretics (%)
3 (14.3)
1 (9.1)
2 (20.0)
0.586
ACEI, Angiotensin-converting enzyme inhibitors; ARNI (Sac/Val), angiotensin receptor-neprilysin inhibitor
(sacubitril/valsartan); ICD, implantable cardioverter deﬁbrillator; LVEF, left ventricular ejection fraction; MRA,
mineralocorticoid receptor antagonist. Data are presented as median (25th and 75th percentiles) or frequency
(percentage); p values refer to between-group differences; bold values refer to statistical signiﬁcance (p ≤ 0.050).
3.2. Group Comparisons
Descriptive group data from the IP and VP, as well as between-phase comparisons,
are presented in Table 2. The median peak work rate during the IP was 55.0 W (46.5;
92.5). The absolute and relative VO2 peak values did not differ between exercise phases
(p = 0.557 and p = 0.400, respectively). RER and VE/VCO2 peak values were lower and
higher, respectively, in the VP than in the IP (p = 0.004 and p = 0.003). The results did not
Int. J. Environ. Res. Public Health 2023, 20, 2764
5 of 10
change when exclusively male or female patients were included in the analyses. Figure 1
shows the Bland-Altman plot for the relative VO2 peak. The mean difference between both
exercise phases was −0.07 mL·kg−1·min−1, while the lower and upper limits of agreement
were −1.59 mL·kg−1·min−1 and 1.45 mL·kg−1·min−1, respectively.
Table 2. Cardiopulmonary responses to the two exercise phases and between-phase comparisons
(n = 21).
Variable
IP
VP
Difference (95% CI)
p
Duration, min
8.5 (7.3; 12.5)
2.8 (2.1; 3.5)
−7.25 (−9.30 to −5.20)
<0.001
HR peak, beats·min−1
112.0 (108.0; 127.0)
109.0 (105.5; 128.0)
−1.00 (−5.92 to 4.92)
0.720
RER peak
1.10 (1.04; 1.12)
1.00 (0.95; 1.08)
−0.08 (−0.13 to −0.02)
0.004
VO2 peak, ml·min−1
1.02 (0.89; 1.59)
1.01 (0.89; 1.58)
−0.007 (−0.026 to 0.012)
0.557
VO2 peak, ml·kg−1·min−1
14.7 (13.1; 17.7)
15.3 (12.7; 17.3)
−0.09 (−0.35 to 0.18)
0.400
O2 pulse, ml·beat−1
10.0 (8.0; 13.0)
10.0 (8.0; 13.0)
0.00 (−0.002 to 0.002)
0.999
VE peak, l·min−1
44.9 (36.4; 64.0)
43.6 (35.8; 62.4)
0.40 (−3.64 to 4.44)
0.550
VE/VO2 peak
36.5 (32.9; 40.9)
35.6 (32.5; 39.1)
0.90 (−2.73 to 4.53)
0.338
VE/VCO2 peak
35.5 (32.4; 36.7)
36.4 (33.4; 39.9)
2.90 (1.33 to 4.47)
0.003
BF peak, breaths·min−1
36.0 (29.5; 41.0)
37.0 (30.5; 41.5)
0.50 (−2.51 to 3.51)
0.746
BF peak, peak breath frequency; CI, conﬁdence interval; HR peak, peak heart rate; IP, incremental phase; O2
pulse, Oxygen pulse; RER peak, peak respiratory exchange ratio, VE peak, peak ventilation; VE/VCO2 peak, peak
ventilatory equivalent for carbon dioxide; VE/VO2 peak, peak ventilatory equivalent for oxygen; VO2 peak, peak
oxygen uptake; VP, veriﬁcation phase. Exercise phase data are presented as median (25th and 75th percentiles);
p values refer to within-subject comparisons (VP vs. IP); bold values refer to statistical signiﬁcance (p ≤ 0.050).
 
Descriptive group data from the IP and VP, as well as between-phase comparisons, 
are presented in Table 2. The median peak work rate during the IP was 55.0 W (46.5; 92.5). 
The absolute and relative VO2 peak values did not differ between exercise phases (p = 
0.557 and p = 0.400, respectively). RER and VE/VCO2 peak values were lower and higher, 
respectively, in the VP than in the IP (p = 0.004 and p = 0.003). The results did not change 
when exclusively male or female patients were included in the analyses. Figure 1 shows 
the Bland-Altman plot for the relative VO2 peak. The mean difference between both 
exercise phases was −0.07 mL·kg−1·min−1, while the lower and upper limits of agreement 
were −1.59 mL·kg−1·min−1 and 1.45 mL·kg−1·min−1, respectively. 
Table 2. Cardiopulmonary responses to the two exercise phases and between-phase comparisons 
(n = 21). 
Variable 
IP 
VP 
Difference (95% CI) 
p 
Duration, min 
8.5 (7.3; 12.5) 
2.8 (2.1; 3.5) 
−7.25 (−9.30 to −5.20) 
<0.001 
HR peak, beats·min−1 
112.0 (108.0; 127.0) 
109.0 (105.5; 128.0) 
−1.00 (−5.92 to 4.92) 
0.720 
RER peak 
1.10 (1.04; 1.12) 
1.00 (0.95; 1.08) 
−0.08 (−0.13 to −0.02) 
0.004 
VO2 peak, ml·min−1 
1.02 (0.89; 1.59) 
1.01 (0.89; 1.58) 
−0.007 (−0.026 to 0.012) 
0.557 
VO2 peak, ml·kg−1·min−1 
14.7 (13.1; 17.7) 
15.3 (12.7; 17.3) 
−0.09 (−0.35 to 0.18) 
0.400 
O2 pulse, ml·beat−1 
10.0 (8.0; 13.0) 
10.0 (8.0; 13.0) 
0.00 (−0.002 to 0.002) 
0.999 
VE peak, l·min−1 
44.9 (36.4; 64.0) 
43.6 (35.8; 62.4) 
0.40 (−3.64 to 4.44) 
0.550 
VE/VO2 peak 
36.5 (32.9; 40.9) 
35.6 (32.5; 39.1) 
0.90 (−2.73 to 4.53) 
0.338 
VE/VCO2 peak 
35.5 (32.4; 36.7) 
36.4 (33.4; 39.9) 
2.90 (1.33 to 4.47) 
0.003 
BF peak, breaths·min−1 
36.0 (29.5; 41.0) 
37.0 (30.5; 41.5) 
0.50 (−2.51 to 3.51) 
0.746 
BF peak, peak breath frequency; CI, confidence interval; HR peak, peak heart rate; IP, incremental 
phase; O2 pulse, Oxygen pulse; RER peak, peak respiratory exchange ratio, VE peak, peak 
ventilation; VE/VCO2 peak, peak ventilatory equivalent for carbon dioxide; VE/VO2 peak, peak 
ventilatory equivalent for oxygen; VO2 peak, peak oxygen uptake; VP, verification phase. Exercise 
phase data are presented as median (25th and 75th percentiles); p values refer to within-subject 
comparisons (VP vs. IP); bold values refer to statistical significance (p ≤ 0.050). 
 
Figure 1. Bland-Altman plot for relative peak oxygen uptake response between the two exercise
phases. Dashed line represents the mean bias, and highlighted zone indices are the limits of agreement
(mean ± 1.96 standard deviation).
3.3. Individual Comparisons
An IP-derived VO2 peak was conﬁrmed (i.e., VO2 max) in 11 (52.4%) patients and not
conﬁrmed (i.e., VO2 peak) in 10 (47.6%). Regarding the patients in whom the VO2 peak
was attained, ﬁve showed higher IP-derived VO2 peak values and ﬁve showed lower IP-
derived VO2 peak values, compared with the VP-derived VO2 peak value. As to the patient
characteristics, comparisons showed that the proportion of smokers was higher (p = 0.035)
in the conﬁrmed group (100%) than in the not conﬁrmed group (60%). Interestingly, the
percentage of female participants did not differ between groups (p = 0.999). Moreover, there
were no between-group differences in any of the remaining analysed variables (p > 0.050)
(see Table 1).
Int. J. Environ. Res. Public Health 2023, 20, 2764
6 of 10
3.4. Conﬁrmed and Not Conﬁrmed Groups
The patients’ responses to both exercise phases in the conﬁrmed and not conﬁrmed
groups can be found in Table S1. Between-phase comparisons showed the same results
as those found when all patients had been included (see Table 2). On the other hand,
between-group comparisons during each exercise phase are shown in Table S2. Although
no statistically signiﬁcant differences were found (p > 0.050), the relative VO2 peak value
was higher in the conﬁrmed group compared to the not conﬁrmed group both in the IP
(2.62 mL·kg−1·min−1 [95%CI = −2.38 to 7.62]; p = 0.305) and the VP (2.35 mL·kg−1·min−1
[95%CI = −2.90 to 7.60]; p = 0.380).
3.5. Agreement between the Traditional and Veriﬁcation Phase Criteria
When the traditional criterion (i.e., RER peak) was used, VO2 max was conﬁrmed in
10 patients and VO2 peak was attained in 11 patients. The median VO2 peak values in
the conﬁrmed and not-conﬁrmed groups during the IP were 16.2 mL·kg−1·min−1 (13.0;
19.9) and 14.1 mL·kg−1·min−1 (13.0; 17.6), while the median values during the VP were
15.7 mL·kg−1·min−1 (13.6; 21.6) and 13.7 mL·kg−1·min−1 (11.5; 16.8), respectively. Regard-
ing the agreement between the two criteria for the determination of VO2 max, there were
10 agreements (47.6%), six false negative cases (28.6%), and ﬁve false positive cases (23.8%).
Moreover, the Kappa index showed that there was no signiﬁcant agreement between both
criteria (Kappa = −0.045; p = 0.583).
4. Discussion
The main objective of this study was to investigate whether the submaximal VP is
a safe and reliable method to validate VO2 max in male and female patients with HFrEF.
To accomplish this, we used both group and individual approaches. Additionally, we
investigated the agreement between the RER and VP criteria to determine VO2 max.
Regarding our results, no adverse events were observed during the exercise tests,
suggesting that VP is a safe method for determining VO2 max in patients with HFrEF. In
agreement with our ﬁndings, Bowen et al. [23] also reported no adverse events in patients
with HFrEF. There is also previous evidence showing that the use of the VP was well-
tolerated in patients with other diseases, such as cancer [29], prehypertension [30], and
metabolic syndrome [31], who are normally sedentary and not familiarised with high-
intensity exercise. Nonetheless, it should be noted that, in the current study, patients who
had a high risk of adverse events (those who presented symptoms of ischaemia or ectopic
heartbeats during the IP) were exempt from performing the VP. Moreover, several patients
had difﬁculty cycling since they were unfamiliar with the cycle ergometer. In this regard,
Manresa-Rocamora et al. [32] reported a greater improvement in the VO2 max after an
exercise-based cardiac rehabilitation programme in studies that conducted the incremental
exercise test on a cycle ergometer compared to studies that used a treadmill in patients with
coronary artery disease. The lack of habituation to the cycle ergometer could explain, in
part, the higher training-induced effect found in these studies, seeing as their baseline VO2
max results were worse than those who used a treadmill. Therefore, a familiarisation period
should be performed before conducting the incremental exercise test to avoid terminating
the test due to peripheral fatigue.
As for the use of individual versus group comparisons for the analysis of VP, con-
tradictory ﬁndings were obtained based on the type of approach used to conduct the
analyses. Group comparisons showed that both exercise phases (i.e., IP and VP) led to
similar median VO2 peak values. These results did not change when only male or female
patients were included in the analysis. Therefore, based on this approach, the VO2 peak
values reached during the IP can be considered as maximal (i.e., VO2 max) in all patients.
This ﬁnding is in line with those of Murias et al. [33] and Bowen et al. [23] in healthy
males and patients with HFrEF, respectively. In this regard, Murias et al. [33], who did
not conduct individual comparisons, concluded that both the submaximal VP (i.e., 85%
of peak power output) and supramaximal VP (i.e., 105% of peak power output) were not
Int. J. Environ. Res. Public Health 2023, 20, 2764
7 of 10
necessary to conﬁrm the VO2 max values reached during the preceding IP. In the same line,
Astorino and Emma [22] and Costa et al. [34], who respectively conducted a review and
a meta-analysis (54 studies), reported no differences in mean VO2 peak values between
the two exercise phases in a sizable number of studies conducted with healthy adults and
individuals with pathology. Previous studies also failed to ﬁnd differences between both
exercise phases in endurance-trained athletes [35,36]. Interestingly, in line with our results,
Costa et al. [34] found that the sex of the participants did not inﬂuence their results and
reported no differences in the aggregate VO2 peak values in male and female participants.
In contrast to these ﬁndings, Moreno-Cabañas et al. [31] and Schaun et al. [37] found higher
mean VO2 peaks during the VP than during the IP in male and female older adults with
obesity and hypertension, respectively. It should be noted that a supramaximal VP (i.e.,
constant load and multistage) preceded by a passive recovery period (i.e., 10–15 min) in the
seated position was conducted, which could explain in part these controversial ﬁndings. In
contrast, Costa et al. [34] reported in their meta-analysis no differences in mean VO2 peaks
regardless of the VP intensity (i.e., submaximal vs. supramaximal), type of recovery (i.e.,
active vs. passive), veriﬁcation timing (i.e., same day vs. different day), and veriﬁcation
phase duration (e.g., less than 80 s) in apparently healthy adults. Bhammar and Chien [30],
who conducted a supramaximal VP, also found no differences in VO2 peak values in adults
with prehypertension. Therefore, our ﬁndings and previous evidence support that the
submaximal VP is not necessary to conﬁrm VO2 max when group comparisons are used,
while the utility of the supramaximal VP, which could lead to controversial ﬁndings, in
patients with HFrEF requires future study. Nonetheless, the achievement of a VO2 peak is
an individual phenomenon and group comparisons may cloud individual differences.
In relation to individual comparisons, our results showed that VO2 max was conﬁrmed
by the VP in 52% of the patients with HFrEF, while a VO2 peak was attained (i.e., the
individual between-phase difference in VO2 peak values higher than 3%) in the remaining
patients (48%). Moreover, the percentage of female patients was the same in the conﬁrmed
and not conﬁrmed groups, suggesting that individual comparisons could be used in both
male and female patients. Nonetheless, the low number of female patients included
warrants future studies to conﬁrm our results. Similarly, Bowen et al. [23], who only
recruited male patients, found that the VO2 max was conﬁrmed in 58% of the patients
with HFrEF included in their study. It should be highlighted that, in contrast to our
study, statistical comparisons between both exercise phases were performed to conduct
individual comparisons and validate the VO2 max. In conclusion, regardless of the criteria
used to carry out individual comparisons, an individual approach should be prioritised to
determine the VO2 max in patients with HFrEF, in accordance with previous reports in the
literature [24,37].
On the other hand, we found no difference in median VO2 peak values between the
two exercise phases in the conﬁrmed and not conﬁrmed groups, which also agrees with the
results of Bowen et al. [23]. In the same line, the current and the former study showed no
difference between the two groups in aggregate VO2 peak values reached during the IP.
However, although statistical signiﬁcance was not reached, both studies showed that the
group VO2 peak values achieved during the IP were higher in the conﬁrmed group (15.9
and 15.1 mL·kg−1·min−1) than in the not conﬁrmed group (13.3 and 13.7 mL·kg−1·min−1).
These ﬁndings seem to support a greater difference in VO2 peak values between both
exercise phases (i.e., VO2 peak attained) in patients with lower cardiorespiratory ﬁtness.
Furthermore, Moreno-Cabañas et al. [31], who included older and less physically ﬁt par-
ticipants with obesity, observed higher VP-derived VO2 peak in 40% of the participants,
while Wood et al. [38], who recruited younger and ﬁtter patients with obesity, only found a
difference in VO2 peak values between the two exercise phases in 15% of the participants.
Therefore, our ﬁndings and previous evidence seem to support that patients with lower
cardiorespiratory ﬁtness may show a greater difference in VO2 peaks between the two
exercise phases, with the use of the VP being even more important for the validation of the
VO2 max in this group of patients.
Int. J. Environ. Res. Public Health 2023, 20, 2764
8 of 10
Finally, regarding the comparison between traditional criteria (i.e., RER) and the VP for
VO2 max determination, we found no agreement between both methods, which is similar
to previous evidence [23]. Interestingly, based on the RER criterion, the VO2 max was
conﬁrmed in ﬁve patients who showed higher VO2 peaks during the VP compared with
the IP (i.e., false positive). There is evidence showing that the RER criterion can be reached
at submaximal intensities (e.g., 80% VO2 max) [11,18], which concurs with our ﬁndings.
Moreover, Bowen et al. [23] showed a direct relationship between RER and workload
increase in patients with HFrEF. In the same line, Moreno-Cabañas et al. [31] observed
that the VO2 plateau was not reliable for determining the VO2 max in participants with
obesity. Therefore, the results of the current and previous studies conﬁrm that traditional
criteria (e.g., RER and VO2 plateau) should not be due to their lack of validity to verify
the VO2 max.
5. Limitations
Some limitations should be mentioned. First, we did not perform a familiarisation
period with the equipment (e.g., cycle ergometer) and, consequently, some patients showed
difﬁculty cycling. Future studies conducted with patients who are sedentary should include
a familiarisation phase before starting the study protocol. Second, there was an uneven sex
distribution among the participants (i.e., 13 males and 8 females). Therefore, to support
our ﬁndings, additional research including more female patients with HFrEF should be
conducted. Third, no prior power analysis was conducted to estimate the optimum number
of patients the study should include.
6. Conclusions
The submaximal VP is a safe and suitable method to determine the VO2 max in
patients with HFrEF. When comparing both exercise phases, an individual approach is
preferable, seeing as aggregate comparisons could mask patients who showed differences
in VO2 peaks between both exercise phases.
Supplementary Materials: The following supporting information can be downloaded at: https:
//www.mdpi.com/article/10.3390/ijerph20042764/s1, Table S1: Cardiopulmonary responses to the
two exercise phases in the conﬁrmed and not conﬁrmed groups, and between-phase comparisons;
Table S2: Between-group comparisons during the two exercise phases.
Author Contributions: Conceptualization, A.M.-R., N.S.-R. and J.M.S.; Methodology, A.M.-R., L.F.-K.,
C.B.-P., N.S.-R., J.M.S. and V.C.-P.; Formal analysis, A.M.-R., L.F.-K. and C.B.-P.; Data curation, A.M.-R.,
L.F.-K. and C.B.-P.; Writing—original draft, A.M.-R., C.B.-P. and J.M.S.; Writing—review & editing,
L.F.-K. and V.C.-P. All authors have read and agreed to the published version of the manuscript.
Funding: This research was funded by the Institute of Health Carlos III (ISCIII, grant number
DTS21/00171, European Commission, FEDER funds) and by the Institute for Health and Biomedical
Research of Alicante (ISABIAL, grant number A2022-0018).
Institutional Review Board Statement: Not applicable.
Informed Consent Statement: Not applicable.
Data Availability Statement: The datasets generated from the current study are available from the
corresponding author upon reasonable request.
Conﬂicts of Interest: The authors declare no conﬂict of interest.
Int. J. Environ. Res. Public Health 2023, 20, 2764
9 of 10
References
1.
Dickstein, K.; Cohen-Solal, A.; Filippatos, G.; McMurray, J.J.; Ponikowski, P.; Poole-Wilson, P.A.; Strömberg, A.; van Veldhuisen, D.J.;
Atar, D.; Hoes, A.W.; et al. ESC guidelines for the diagnosis and treatment of acute and chronic heart failure 2008: The Task Force for
the diagnosis and treatment of acute and chronic heart failure 2008 of the European Society of Cardiology. Developed in collaboration
with the Heart Failure Association of the ESC (HFA) and endorsed by the European Society of Intensive Care Medicine (ESICM). Eur.
J. Heart Fail 2008, 10, 933–989. [CrossRef] [PubMed]
2.
Members, W.C.; Hunt, S.A.; Abraham, W.T.; Chin, M.H.; Feldman, A.M.; Francis, G.S.; Ganiats, T.G.; Jessup, M.; Konstam, M.A.;
Mancini, D.M.; et al. 2009 focused update incorporated into the ACC/AHA 2005 Guidelines for the Diagnosis and Management
of Heart Failure in Adults: A report of the American College of Cardiology Foundation/American Heart Association Task Force
on Practice Guidelines: Developed in collaboration with the International Society for Heart and Lung Transplantation. Circulation
2009, 119, e391–e479. [CrossRef] [PubMed]
3.
McMurray, J.J.; Adamopoulos, S.; Anker, S.D.; Auricchio, A.; Böhm, M.; Dickstein, K.; Falk, V.; Filippatos, G.; Fonseca, C.;
Gomez-Sanchez, M.A.; et al. ESC Guidelines for the diagnosis and treatment of acute and chronic heart failure 2012: The Task
Force for the Diagnosis and Treatment of Acute and Chronic Heart Failure 2012 of the European Society of Cardiology. Developed
in collaboration with the Heart Failure Association (HFA) of the ESC. Eur. Heart J. 2012, 33, 1787–1847. [CrossRef] [PubMed]
4.
Mancini, D.M.; Eisen, H.; Kussmaul, W.; Mull, R.; Edmunds, L.H.; Jr Wilson, J.R. Value of peak exercise oxygen consumption for
optimal timing of cardiac transplantation in ambulatory patients with heart failure. Circulation 1991, 83, 778–786. [CrossRef]
[PubMed]
5.
Myers, J.; Gullestad, L.; Vagelos, R.; Do, D.; Bellin, D.; Ross, H.; Fowler, M.B. Cardiopulmonary exercise testing and prognosis in
severe heart failure: 14 mL/kg/min revisited. Am. Heart J. 2000, 139 (Pt 1), 78–84. [CrossRef]
6.
Francis, D.; Shamim, W.; Davies, L.C.; Piepoli, M.; Ponikowski, P.; Anker, S.; Coats, A. Cardiopulmonary exercise testing for
prognosis in chronic heart failure: Continuous and independent prognostic value from VE/VCO(2)slope and peak VO(2). Eur.
Heart J. 2000, 21, 154–161. [CrossRef] [PubMed]
7.
Wagner, P.D. Determinants of maximal oxygen transport and utilization. Annu. Rev. Physiol. 1996, 58, 21–50. [CrossRef]
8.
O’Neill, J.O.; Young, J.B.; Pothier, C.E.; Lauer, M.S. Peak oxygen consumption as a predictor of death in patients with heart failure
receiving beta-blockers. Circulation 2005, 111, 2313–2318. [CrossRef]
9.
Ross, R.; Blair, S.N.; Arena, R.; Church, T.S.; Després, J.P.; Franklin, B.A.; Haskell, W.L.; Kaminsky, L.A.; Levine, B.D.;
Lavie, C.J.; et al. Importance of Assessing Cardiorespiratory Fitness in Clinical Practice: A Case for Fitness as a Clinical Vi-
tal Sign: A Scientiﬁc Statement From the American Heart Association. Circulation 2016, 134, e653–e699. [CrossRef]
10.
Astorino, T.A. Alterations in VOmax and the VO plateau with manipulation of sampling interval. Clin. Physiol. Funct. Imaging
2009, 29, 60–67. [CrossRef]
11.
Poole, D.C.; Jones, A.M. Measurement of the maximum oxygen uptake
.
Vo(2max):
.
Vo(2peak) is no longer acceptable. J. Appl.
Physiol. 2017, 122, 997–1002. [CrossRef]
12.
Midgley, A.W.; McNaughton, L.R.; Polman, R.; Marchant, D. Criteria for determination of maximal oxygen uptake: A brief
critique and recommendations for future research. Sports Med. 2007, 37, 1019–1028. [CrossRef] [PubMed]
13.
Poole, D.C.; Wilkerson, D.P.; Jones, A.M. Validity of criteria for establishing maximal O2 uptake during ramp exercise tests. Eur. J.
Appl. Physiol. 2008, 102, 403–410. [CrossRef] [PubMed]
14.
Niemelä, K.; Palatsi, I.; Linnaluoto, M.; Takkunen, J. Criteria for maximum oxygen uptake in progressive bicycle tests. Eur. J. Appl.
Physiol. Occup. Physiol. 1980, 44, 51–59. [CrossRef] [PubMed]
15.
Day, J.R.; Rossiter, H.B.; Coats, E.M.; Skasick, A.; Whipp, B.J. The maximally attainable VO2 during exercise in humans: The peak
vs. maximum issue. J. Appl. Physiol. 2003, 95, 1901–1907. [CrossRef]
16.
Howley, E.T.; Bassett, D.R., Jr.; Welch, H.G. Criteria for maximal oxygen uptake: Review and commentary. Med. Sci. Sports Exerc.
1995, 27, 1292–1301. [CrossRef]
17.
Mezzani, A.; Agostoni, P.; Cohen-Solal, A.; Corrà, U.; Jegier, A.; Kouidi, E.; Mazic, S.; Meurin, P.; Piepoli, M.; Simon, A.; et al.
Standards for the use of cardiopulmonary exercise testing for the functional evaluation of cardiac patients: A report from the
Exercise Physiology Section of the European Association for Cardiovascular Prevention and Rehabilitation. Eur. J. Cardiovasc.
Prev. Rehabil. 2009, 16, 249–267. [CrossRef]
18.
Beltz, N.M.; Gibson, A.L.; Janot, J.M.; Kravitz, L.; Mermier, C.M.; Dalleck, L.C. Graded Exercise Testing Protocols for the
Determination of VO(2)max: Historical Perspectives, Progress, and Future Considerations. J. Sports Med. 2016, 2016, 3968393.
[CrossRef]
19.
Midgley, A.W.; Carroll, S. Emergence of the veriﬁcation phase procedure for conﬁrming ’true’ VO(2max). Scand. J. Med. Sci.
Sports 2009, 19, 313–322. [CrossRef]
20.
Schaun, G.Z. The Maximal Oxygen Uptake Veriﬁcation Phase: A Light at the End of the Tunnel? Sports Med. Open 2017, 3, 44.
[CrossRef]
21.
Villanueva, I.R.; Campbell, J.C.; Medina, S.M.; Jorgensen, T.M.; Wilson, S.L.; Angadi, S.S.; Gaesser, G.A.; Dickinson, J.M.
Comparison of constant load exercise intensity for veriﬁcation of maximal oxygen uptake following a graded exercise test in
older adults. Physiol. Rep. 2021, 9, e15037. [CrossRef]
22.
Astorino, T.A.; Emma, D. Utility of Veriﬁcation Testing to Conﬁrm Attainment of Maximal Oxygen Uptake in Unhealthy
Participants: A Perspective Review. Sports 2021, 9, 9080108. [CrossRef] [PubMed]
Int. J. Environ. Res. Public Health 2023, 20, 2764
10 of 10
23.
Bowen, T.S.; Cannon, D.T.; Begg, G.; Baliga, V.; Witte, K.K.; Rossiter, H.B. A novel cardiopulmonary exercise test protocol and
criterion to determine maximal oxygen uptake in chronic heart failure. J. Appl. Physiol. 2012, 113, 451–458. [CrossRef]
24.
Noakes, T.D. Maximal oxygen uptake as a parametric measure of cardiorespiratory capacity: Comment. Med. Sci. Sports Exerc.
2008, 40, 585–586. [CrossRef] [PubMed]
25.
Dixon, P.M.; Saint-Maurice, P.F.; Kim, Y.; Hibbing, P.; Bai, Y.; Welk, G.J. A Primer on the Use of Equivalence Testing for Evaluating
Measurement Agreement. Med. Sci. Sports Exerc. 2018, 50, 837–845. [CrossRef] [PubMed]
26.
Lamarra, N.; Whipp, B.J.; Ward, S.A.; Wasserman, K. Effect of interbreath ﬂuctuations on characterizing exercise gas exchange
kinetics. J. Appl. Physiol. 1987, 62, 2003–2012. [CrossRef]
27.
Kaiser, J. An exact and a Monte Carlo proposal to the Fisher–Pitman permutation tests for paired replicates and for independent
samples. Stata J. 2007, 7, 402–412. [CrossRef]
28.
Campbell, M.J.; Gardner, M.J. Calculating conﬁdence intervals for some non-parametric analyses. Br. Med. J. (Clin. Res. Ed.) 1988,
296, 1454–1456. [CrossRef]
29.
Schneider, J.; Schlüter, K.; Wiskemann, J.; Rosenberger, F. Do we underestimate maximal oxygen uptake in cancer survivors?
Findings from a supramaximal veriﬁcation test. Appl. Physiol. Nutr. Metab. 2020, 45, 486–492. [CrossRef]
30.
Bhammar, D.M.; Chien, L.C. Quantiﬁcation and Veriﬁcation of Cardiorespiratory Fitness in Adults with Prehypertension. Sports
2021, 9, 9010009. [CrossRef]
31.
Moreno-Cabañas, A.; Ortega, J.F.; Morales-Palomo, F.; Ramirez-Jimenez, M.; Mora-Rodriguez, R. Importance of a veriﬁcation test
to accurately assess
.
VO(2) max in unﬁt individuals with obesity. Scand J. Med. Sci. Sports 2020, 30, 583–590. [CrossRef] [PubMed]
32.
Manresa-Rocamora, A.; Sarabia, J.M.; Sánchez-Meca, J.; Oliveira, J.; Vera-Garcia, F.J.; Moya-Ramón, M. Are the Current Cardiac
Rehabilitation Programs Optimized to Improve Cardiorespiratory Fitness in Patients? A Meta-Analysis. J. Aging Phys. Act. 2021,
29, 327–342. [CrossRef] [PubMed]
33.
Murias, J.M.; Pogliaghi, S.; Paterson, D.H. Measurement of a True [Formula: See text]O(2max) during a Ramp Incremental Test Is
Not Conﬁrmed by a Veriﬁcation Phase. Front. Physiol. 2018, 9, 143. [CrossRef] [PubMed]
34.
Costa, V.A.B.; Midgley, A.W.; Carroll, S.; Astorino, T.A.; de Paula, T.; Farinatti, P.; Cunha, F.A. Is a veriﬁcation phase useful for
conﬁrming maximal oxygen uptake in apparently healthy adults? A systematic review and meta-analysis. PLoS ONE 2021, 16,
e0247057. [CrossRef] [PubMed]
35.
Midgley, A.W.; Carroll, S.; Marchant, D.; McNaughton, L.R.; Siegler, J. Evaluation of true maximal oxygen uptake based on a
novel set of standardized criteria. Appl. Physiol. Nutr. Metab. 2009, 34, 115–123. [CrossRef]
36.
Foster, C.; Kuffel, E.; Bradley, N.; Battista, R.A.; Wright, G.; Porcari, J.P.; Lucia, A.; Dekoning, J.J. VO2max during successive
maximal efforts. Eur. J. Appl. Physiol. 2007, 102, 67–72. [CrossRef]
37.
Schaun, G.Z.; Alberton, C.L.; Gomes, M.L.B.; Santos, L.P.; Bamman, M.M.; Mendes, G.F.; Häfele, M.S.; Andrade, L.S.; Alves, L.;
DE Ataides, V.A.; et al. Maximal Oxygen Uptake Is Underestimated during Incremental Testing in Hypertensive Older Adults:
Findings from the HAEL Study. Med. Sci. Sports Exerc. 2021, 53, 1452–1459. [CrossRef]
38.
Wood, R.E.; Hills, A.P.; Hunter, G.R.; King, N.A.; Byrne, N.M. Vo2max in overweight and obese adults: Do they meet the threshold
criteria? Med. Sci. Sports Exerc. 2010, 42, 470–477. [CrossRef]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual
author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to
people or property resulting from any ideas, methods, instructions or products referred to in the content.

Is the Verification Phase a Suitable Criterion for the Determination of Maximum Oxygen Uptake in Patients with Heart Failure and Reduced Ejection Fraction? A Validation Study.
Manresa-Rocamora, Agustín,Fuertes-Kenneally, Laura,Blasco-Peris, Carles,Sempere-Ruiz, Noemí,Sarabia, José Manuel,Climent-Paya, Vicente