All relevant data are within the manuscript and its Supporting Information files.

Introduction {#sec001}
============

Effective temperature-controlled storage is vital to preserving vaccine potency. In the United States, most vaccines must be maintained at temperatures between 2 °C and 8 °C from the point of product manufacture until delivery to a patient \[[@pone.0235777.ref001]\]. Prolonged and extreme excursions from this range can compromise the efficacy of the vaccine. Most of these vaccines are freeze-sensitive, as exposure to temperatures below 0 °C induces morphological changes in the antigens and/or adjuvants, leading to irrevocable potency losses \[[@pone.0235777.ref002]--[@pone.0235777.ref004]\].

Present-day Centers for Disease Control (CDC) guidance permits the use of domestic refrigerator units for vaccine storage \[[@pone.0235777.ref005]\]. While these units are designed and marketed for home food storage, their low cost, availability, and large storage capacity make them attractive purchases for vaccine providers. Domestic refrigerator units may be used to safely maintain stored vaccines when paired with proper user training and temperature monitoring techniques \[[@pone.0235777.ref006],[@pone.0235777.ref007]\]. However, field studies in the United States, Australia, and Tunisia documented repeated exposures to temperatures outside the permissible range in domestic refrigerators used for vaccine storage, and the widespread use of domestic refrigerators is frequently cited as a cause of vaccine wastage \[[@pone.0235777.ref008]--[@pone.0235777.ref012]\].

Prior studies indicated that placing added thermal mass in the form of water bottles inside a vaccine storage refrigerator may improve temperature stability and prolong viable vaccine storage temperatures during adverse events such as power outages, defrost cycle activation, and routine door opening \[[@pone.0235777.ref006],[@pone.0235777.ref007]\]. Current CDC guidelines recommend storing water bottles inside domestic refrigerators to prevent vaccine storage in unsuitable regions (like the refrigerator door), and to act as a "thermal ballast," stabilizing vaccine temperatures and minimizing excursions \[[@pone.0235777.ref005]\]. However, the quantity of thermal ballast materials needed to improve the stability of stored vaccine temperatures has not been thoroughly studied or documented, so existing recommendations are subject to interpretation by users and immunization program administrators.

In this study, we examined the effect of thermal ballast loads on the temperature stability of vaccine stored in two domestic refrigerators under normal operation and during adverse events. Our primary objective was to determine the minimum quantity of thermal ballast materials required to reduce the incidence or severity of temperature excursions, as compared to the same units used without thermal ballast materials.

Each refrigerator was tested using four ballast loads, in which 25%, 17%, 8%, or 0% of the total refrigerator storage volume was filled with water bottles. Stored vaccine temperatures were monitored during closed-door, steady-state operation, defrost cycle activation, repeated door opening, and power loss events, to assess the impact of each load percentage on the stored product temperature. In addition, different storage containers (open plastic bins, covered plastics bins, and metal trays) were included to investigate whether the temperature stability of refrigerated vaccine is correlated with storage container type.

Materials and methods {#sec002}
=====================

Refrigerator selection {#sec003}
----------------------

Two domestic refrigerator units were included in this study: a combination refrigerator/freezer, and a standalone refrigerator. While both designs are marketed as consumer-grade, food storage refrigerators, these types of units are frequently used for vaccine storage due to their low cost and availability \[[@pone.0235777.ref011]--[@pone.0235777.ref014]\]. In the 2019 Vaccine Storage and Handling Toolkit \[[@pone.0235777.ref005]\], the CDC recommends storing vaccine in a purpose-built refrigerator designed specifically for pharmaceutical storage. However, if a purpose-built unit is not available, a domestic standalone unit is suggested as an acceptable alternative. Domestic combination refrigerator/freezer units are generally not recommended for vaccine storage due to an increased risk of accidental freezing. However, current guidance permits the use domestic combination units for refrigerated vaccine storage in the absence of recommended alternatives \[[@pone.0235777.ref005]\]. Since domestic refrigerators are designed for food storage, these units are typically not engineered to match the level of temperature control and stability provided by a purpose-built vaccine storage refrigerator. In fact, meta-analyses of domestic food storage refrigerator temperature studies from 1979 to 2016 indicate that household units used worldwide routinely operate outside the recommended refrigerated food storage temperature range of 0 °C to 8 °C \[[@pone.0235777.ref015],[@pone.0235777.ref016]\].

Both units used in this study featured an upright, swinging door design. The combination unit had three glass storage shelves inside its refrigerated compartment, and the standalone unit had four interior wire shelves. The standalone unit was also tested with an aftermarket glass shelf installation as a separate case study, described in later sections of this article. Both units were housed in the National Institute of Standards and Technology (NIST) Gaithersburg Thermal Studies Laboratory throughout the study. Refrigerator details are summarized in [Table 1](#pone.0235777.t001){ref-type="table"}.

10.1371/journal.pone.0235777.t001

###### Domestic test refrigerator attributes.

![](pone.0235777.t001){#pone.0235777.t001g}

                                                    Combination Refrigerator/Freezer   Standalone Refrigerator
  ------------------------------------------------- ---------------------------------- -------------------------------
  **Model**[^a^](#t001fn001){ref-type="table-fn"}   Kenmore 253.78179801               Kenmore 253.60722
  **Automatic Defrost**                             Yes                                Yes
  **Freezer**                                       Yes, top freezer                   No
  **Refrigerated compartment capacity**:            0.40 m^3^ (14.13 ft^3^)            0.47 m^3^ (16.7 ft^3^)
  **Shelf material (quantity)**                     Glass (3 shelves)                  Resin-coated wire (4 shelves)
  **Electrical rating**                             4.5 A / 115 V / 60 Hz              5 A / 115 V / 60 Hz
  **Refrigerant**                                   R134a                              R134a
  **Manufacture date**                              November 2009                      May 2009

^a^ Commercial equipment is identified to specify the experimental procedure adequately. Such identification is not intended to imply recommendation or endorsement by the National Institute of Standards and Technology, nor is it intended to imply that the equipment identified is necessarily the best available for the purpose.

Both refrigerators featured an analog temperature control dial, which was kept on a midpoint setting throughout testing. This corresponded to a nominal temperature set point of 4 °C to 5 °C in each refrigerator.

Vaccine storage setup {#sec004}
---------------------

Both refrigerators were outfitted with storage containers arranged in the center of each unit, filling approximately 50% of the available shelf space. The top shelf, floor, door, and approximately 15 cm of space adjacent to each interior side wall was kept free of stored vaccines throughout the study to allow for placement of water bottles in these areas. Each of the remaining shelves was outfitted with two closed plastic bins, one large metal tray, and two uncovered plastic bins ([Fig 1](#pone.0235777.g001){ref-type="fig"}). The storage containers were filled with vaccine vials and pre-filled syringes stored in their original, manufacturer-supplied cardboard packaging, along with unboxed vaccine vials placed directly in the bin or tray. Two vaccines from each container, one boxed and one unboxed, were selected for temperature monitoring ([Fig 2](#pone.0235777.g002){ref-type="fig"}). Vaccine and storage container positions remained fixed throughout the study. The vaccines used in this study included a mix of 0.5 mL single dose vials, 0.5 mL prefilled syringes, and 5 mL multidose vials of common U.S. childhood vaccines designated for refrigerated storage between 2 °C and 8 °C, such as influenza, diphtheria and tetanus, hepatitis A and B, pneumococcal disease, and haemophilus influenzae type b virus.

![Stored vaccine setup.\
Each shelf was outfitted with five storage containers (left to right): two closed plastic bins, one large metal tray, and two uncovered plastic bins. The storage containers were filled with a mix of boxed and unboxed vaccines. The top shelf (not shown) was kept free of stored vaccines to allow for water bottle placement in this area.](pone.0235777.g001){#pone.0235777.g001}

![Monitored vaccine vial setup.\
Three boxed vaccines (shown in yellow) and three unboxed vaccine vials (shown in pink), kept in bins as shown on each shelf, were outfitted with thermocouple probes for temperature monitoring during the study. Additional, unmonitored vaccine boxes and vials, not shown in this illustration, were used to fill each bin (See [Fig 1](#pone.0235777.g001){ref-type="fig"}).](pone.0235777.g002){#pone.0235777.g002}

Temperature monitoring {#sec005}
----------------------

A total of twenty-nine Type T thermocouples were used to monitor vaccines in the two refrigerators, with twelve monitored vaccines in the combination unit, and seventeen monitored vaccines in the standalone unit. Each thermocouple sensing junction was immersed in the 0.5 mL vaccine fluid volume contained in a vial or pre-filled syringe. Thermocouple reference junctions were maintained in a circulating water bath controlled to 23 °C ± 0.005 °C. Prior to their use in the study, each thermocouple was calibrated at the melting point of ice (0 °C) by immersing the thermocouple sensing junction in a Dewar flask containing crushed ice and distilled water, as described in NIST Technical Note 1411 \[[@pone.0235777.ref017]\]. The ice point measurements were used to generate a temperature offset for each thermocouple, which was applied to all measurements collected during the study. Thermocouple voltages were read via automated 6 ½ digit multimeter data acquisition units, which were set to scan and log readings to a computer every 10 seconds. The uncertainty of the temperature measurement system is shown in [Table 2](#pone.0235777.t002){ref-type="table"}.

10.1371/journal.pone.0235777.t002

###### Temperature measurement system uncertainties[^a^](#t002fn001){ref-type="table-fn"}.

![](pone.0235777.t002){#pone.0235777.t002g}

  ---------------------------------------------------------- -----------
  **Type A uncertainties**                                   
                                                             u~i~ / °C
  Thermocouple accuracy                                      0.01
  Thermocouple repeatability                                 0.05
  Total A                                                    0.05
  **Type B uncertainties**, **(rectangular distribution)**   
  Multimeter accuracy                                        0.01
  Multimeter resolution                                      0.0005
  Thermocouple reproducibility                               0.09
  Reference junction bath stability                          0.005
  Total B                                                    0.05
  Total Standard Uncertainty (k = 1)                         0.07
  **Total Expanded Uncertainty (k = 2)**                     **0.15**
  ---------------------------------------------------------- -----------

^a^Measurement uncertainties calculated according to the Guidelines for Evaluating and Expressing the Uncertainty of NIST Measurement Results \[[@pone.0235777.ref018]\].

Thermal ballast {#sec006}
---------------

Each refrigerator was loaded and tested with varying quantities of water bottles to assess the effect of thermal ballast loading on temperature stability. Commercially available plastic drinking water bottles in the ubiquitous 500 mL (16.9 fl oz) size were used throughout the study. Larger commercially available plastic bottles and jugs were also included in some of the larger thermal ballast loads, ranging from 1000 mL (33.8 fl oz) bottles to 3950 mL (1.04 gal) jugs. The total water bottle volume used for each ballast load percentage was calculated based on the manufacturer-specified storage volume of the refrigerator under test. Water bottles were distributed in accordance with vaccine management guidelines issued by the Centers for Disease Control and Prevention (CDC), with the bulk of the bottles concentrated in the refrigerator door, on the top shelf, and floor level of the unit \[[@pone.0235777.ref005]\]. The relative distribution of water bottles was kept consistent for each of the 8%, 17%, and 25% ballast load configurations. In each configuration, approximately 27% of total bottle volume was in stored in the refrigerator door, 30% on the refrigerator floor, 23% on the top shelf, and the remaining 20% distributed near the walls of the intermediate shelves. [Fig 3](#pone.0235777.g003){ref-type="fig"} shows this distribution applied to a 10% ballast load in the standalone unit. Additional photos of 25% and 8% thermal ballast loads are shown in [S1 Fig](#pone.0235777.s001){ref-type="supplementary-material"}.

![Standalone refrigerator filled with a 10% thermal ballast load comprised of 500 mL (16.9 oz) plastic drinking water bottles.\
Water bottles were concentrated in door, top shelf, and floor level of the unit, with a small number of bottles placed along the side walls of the unit. The relative distribution of thermal ballast (water bottles), detailed in the text, was kept fixed throughout the study. Plastic and metal bins placed in the center of the unit were used to contain a mix of boxed and unboxed vaccine vials.](pone.0235777.g003){#pone.0235777.g003}

Test protocol {#sec007}
-------------

The test protocol was designed to probe the effect of varying thermal ballast loads on refrigerator performance during normal operating conditions and typical adverse events. "Normal operation" test modes included 1) closed door operation and 2) repeated door opening. The closed-door operation test was executed for a minimum of 24 h, to ensure capture of at least one defrost cycle event. In the repeated door opening test, the refrigerator door was held open for 30 s at 5 min intervals, over a period of 2 h, to simulate frequent vaccine retrieval during a busy clinic workday. The "adverse events" examined in this study were 3) door left open at an angle of 90° for 1 h, and 4) a power outage event, in which the refrigerator was unplugged and monitored for 24 h. The four-test sequence was replicated for each thermal ballast load (25%, 17%, 8%, 0%).

Both refrigerator units were tested according to this protocol. Vaccines and thermocouples were arranged in each unit prior to testing and kept fixed throughout the study. The maximum thermal ballast load (25%) was arranged in the unit prior to the first test, and temperatures were monitored for a minimum of 24 h to ensure that steady state operating conditions were achieved before testing commenced. Following completion of the four-test sequence, water bottles were removed from the unit to achieve the second-largest ballast load (17%), and again, temperatures were monitored for a minimum of 24 h to ensure equilibration before proceeding with the next test sequence. This process was repeated for the remaining ballast loads (8%, 0%).

Shelf material case study {#sec008}
-------------------------

We also investigated possible effects of refrigerator shelf construction on vaccine storage conditions in a related case study. In this investigation, the standalone refrigerator was outfitted with vaccine-filled trays as before, and vaccine vials distributed within the storage trays were monitored with thermocouple probes to map out temperature conditions inside the refrigerator storage volume, as shown in Figs [1](#pone.0235777.g001){ref-type="fig"} and [2](#pone.0235777.g002){ref-type="fig"}.

In the first set of trials, the trays were placed, as before, directly on the refrigerator's resin-coated wire shelves. In the second set of trials, glass sheets were cut to match the dimensions of the existing wire shelves, and then installed on top of the wire shelves, with two sheets placed side by side on each shelf. Sheets were cut from consumer grade clear replacement glass. Final dimensional specifications are listed in [Table 3](#pone.0235777.t003){ref-type="table"}.

10.1371/journal.pone.0235777.t003

###### Glass sheet specifications.

![](pone.0235777.t003){#pone.0235777.t003g}

  Installation location       Length (cm)   Width (cm)   Thickness (cm)
  --------------------------- ------------- ------------ ----------------
  **Shelves 1 (top), 2, 3**   40.5          30.5         0.238
  **Shelf 4 (bottom)**        35.5          30.5         0.238

The installed glass sheets were positioned to preserve an air gap between the refrigerator walls and the edges of the shelves to avoid completely obstructing the refrigerator airflow. The air gap between the back wall of the refrigerator and the installed shelves was approximately 1 cm, and the air gap between the side walls and the adjacent edge of each shelf was approximately 3.5 cm. After installation, vaccine storage trays were placed on the glass shelves, following the same loading pattern used in the thermal ballast trials.

Each of the setups described above was subjected to a power outage test and a repeated door opening test, utilizing the same procedures described in the Thermal Ballast Test Protocol subsection. No thermal ballast materials were placed in the unit during the shelf material case study tests.

Evaluation methods {#sec009}
------------------

Each measurement trial in the described in the preceding sections yielded a time-temperature data series of twelve or seventeen temperature sensors, with data points every 10 s for periods of hours up to several days. To compare and draw meaningful conclusions from these data sets, we summarized the results from each trial according to the methods described in this section.

While temperature cycling is normal and expected in domestic refrigerators, very large fluctuations can result in temperatures outside recommended vaccine storage temperature limits. Since the recommended set point for refrigerated vaccine storage is 5 °C, a unit that routinely cycles through a range of 3 °C or more may subject stored vaccines to excursions outside 2 °C to 8 °C. As a result, the temperature range, Δ*T* = *T*~*max*~ − *T*~*min*~, recorded by each monitored vaccine during each trial, is taken as an indicator of thermal stability. A simple mean of the Δ*T* for all monitored vials is given by $\overline{\Delta T}$. We therefore define $\overline{\Delta T} > 3{^\circ}C$ as the condition for a vaccine temperature excursion, and $\overline{\Delta T} < 3{^\circ}C$ as the condition for proper vaccine temperature maintenance. Defining temperature excursions in this way allows us to separate our evaluation from small shifts in absolute refrigerator set point temperature.

Similarly, we define "viable vaccine storage time" as the duration of time in which vaccines were maintained within 3 °C of their initial storage temperature $(\overline{\Delta T} < 3{^\circ}C)$ during a power loss or door left open event. In this framework, a temperature increase of more than 3 °C is equivalent to the start of a temperature excursion, and the end of viable vaccine storage time.

In the defrost cycle and door opening trials, $\overline{\Delta T}$ is used to assess refrigerator stability at different ballast loading levels. In the door left open and power outage trials, viable vaccine storage time is used as the primary assessment parameter.

In addition, a Pearson correlation coefficient, *r*, was calculated assess the strength of the linear relationship between thermal ballast load percentage and viable vaccine storage time during power outage trials. The formula for *r* is the covariance of the two data sets divided by the product of their standard deviations: $$r = \frac{\sum\left( {x - \overline{x}} \right)(t - \overline{t})}{\sqrt{\sum\left( {x - \overline{x}} \right)^{2}\mspace{360mu}\sum{(t - \overline{t})}^{2}}}$$ Here, *x* denotes the thermal ballast load percentage for a measurement, *t* denotes the viable vaccine storage time for a particular measured vial in a particular trial, and $\overset{¯}{x}$, $\overset{¯}{t}$ are the sample means of these variables, calculated from all sensor outputs in all four trials conducted in each unit.

Results {#sec010}
=======

Defrost cycle {#sec011}
-------------

Normal refrigerator operation causes periodic temperature cycling of stored vaccines, even when the door is kept closed. The magnitude and frequency of stored vaccine temperature fluctuations necessarily varies by refrigerator model, temperature setting, stored vaccine load, and environmental operating conditions. Larger, transient fluctuations typically occur during defrost cycle activation and the subsequent temperature recovery period. Basic domestic refrigerators periodically initiate a defrost cycle that includes operation of a heater attached to the evaporator coils \[[@pone.0235777.ref018],[@pone.0235777.ref019]\]. This process melts accumulated frost from the coils, and as a side effect, produces a transient rise in cabinet temperature. Once the defrost heater deactivates, the refrigerator operates the cooling cycle for an extended period to cool the unit back to its set point. During the recovery phase, the cabinet temperature may drop below the setpoint before the compressor cycles off. In both units tested in this study, a measurable, transient temperature rise associated with defrost cycle activation occurred approximately once every 24 h. An example of the defrost cycle temperature response in the standalone refrigerator used in this study is shown in [S1 Fig](#pone.0235777.s001){ref-type="supplementary-material"}.

The defrost cycle data are summarized as described in the Evaluation methods, and the results are displayed in [Fig 4](#pone.0235777.g004){ref-type="fig"}. $\overline{\Delta T} > 3{^\circ}C$ indicates a condition likely to result in vaccine temperature excursions.

![Refrigerator cabinet temperature changes during defrost cycle activation at varying thermal ballast load percentages.\
Round blue markers indicate the average temperature rise of all monitored vaccines during each trial, and error bars show the range of these measurements, corresponding to the within-trial variation among monitored vaccines.](pone.0235777.g004){#pone.0235777.g004}

In the standalone unit, the $\overline{\Delta T} > 3{^\circ}C$ condition occurred during one trial, in which no thermal ballast was stored inside the refrigerator. Depending on the refrigerator setpoint, a fluctuation of this magnitude could expose stored vaccine to a small temperature excursion. In this unit, filling 25% of the refrigerator storage volume with water bottles reduced defrost cycle-induced temperature fluctuations by two-thirds. A more practical 8% thermal ballast load reduced temperature cycling by one-half, such that all monitored vaccine temperatures were maintained within the 3 °C excursion threshold.

In the combination unit, no $\overline{\Delta T} > 3{^\circ}C$ conditions occurred during any of the defrost cycle trials. Filling 25% of the refrigerator volume with water bottles reduced temperature fluctuations by 70%. However, smaller ballast loads had little effect on the defrost cycle temperature response. As shown in [Fig 4](#pone.0235777.g004){ref-type="fig"}, temperature cycling was nearly identical for the 0%, 8%, and 17% loads.

Repeated door opening {#sec012}
---------------------

Repeated door opening in vaccine refrigerators also contributes to thermal cycling of vaccines \[[@pone.0235777.ref006],[@pone.0235777.ref007],[@pone.0235777.ref020]\]. The results of the repeated door opening trials are shown in [Fig 5](#pone.0235777.g005){ref-type="fig"}.

![Refrigerator cabinet temperature changes during repeated door opening trials performed with varying thermal ballast load percentages.\
Round blue markers indicate the average temperature rise of all monitored vaccines during each trial, and error bars show the range of these measurements, corresponding to the within-trial variation among monitored vaccines.](pone.0235777.g005){#pone.0235777.g005}

In the standalone unit, the average vaccine temperature variation exceeded 3 °C during all trials, regardless of thermal ballast load. While the temperature response varied between trials, the within-trial variation across all monitored vaccines, as indicated by error bars in [Fig 5](#pone.0235777.g005){ref-type="fig"} (right), exceeded the trial-to-trial variation in all but the 8% thermal ballast trial. Within-trial variations may be attributed to thermal gradients inside the refrigerator, which increase with repeated door opening \[[@pone.0235777.ref021]\]. Adding thermal ballast to the standalone refrigerator does not appear to mitigate temperature instability caused by repeated door opening events, as vaccine temperature excursions occurred at all ballast loading levels tested during the repeated door opening trials.

By contrast, increased thermal ballast load appears to be correlated with improved temperature stability in a combination unit at higher ballast percentages (18%, 25%) subjected to repeated door opening. At these higher load percentages, the average temperature variation dropped below the 3 °C threshold. During the 25% thermal ballast load door opening trial, temperature fluctuations were controlled to within 2.1 °C or less across all monitored vials.

Door open for one hour {#sec013}
----------------------

Both units were also evaluated with various ballast loads to determine the impact on temperature stability during a worst-case test, in which the refrigerator door was kept open at an angle of 90° for one hour. In both units, vaccines were maintained within proper temperature storage conditions for an average of 14 minutes after the door was opened, regardless of the thermal ballast load used ([S5](#pone.0235777.s007){ref-type="supplementary-material"} and [S6](#pone.0235777.s008){ref-type="supplementary-material"} Tables). The one exception to this occurred during the 25% ballast trial performed in the combination refrigerator. In this case, vaccines remained within a viable storage temperature range for an average of 22 minutes ([S5 Table](#pone.0235777.s007){ref-type="supplementary-material"}).

Power outage {#sec014}
------------

Electrical power outages present a significant threat to refrigerated vaccines stored in the absence of a backup power source \[[@pone.0235777.ref022]--[@pone.0235777.ref024]\]. The U.S. Energy Information Administration (EIA) publishes reliability data provided by domestic utility companies, including frequency and duration of power interruptions experienced per customer each year \[[@pone.0235777.ref025]\]. As summarized in a recent EIA article \[[@pone.0235777.ref026]\], U.S. customers experienced, on average, cumulative power interruptions of 7.8 h in 2017, based on the utility-reported System Average Interruption Duration Index (SAIDI). The EIA reports that this figure represents a significant increase from the 2016 SAIDI total of 4.2 h, and may be attributed to the increased incidence of major events, such as hurricanes and winter storms, in 2017. In Maine and Florida, hurricanes and storms resulted in average electrical power interruption durations as long as 42 h in 2017 \[[@pone.0235777.ref026]\].

The frequency and duration of power disruptions in the United States are projected to increase due to the nation's deteriorating power grid infrastructure \[[@pone.0235777.ref027],[@pone.0235777.ref028]\], making power loss mitigation strategies an ever-more critical component of a safe and effective vaccine management program. Prior tests indicated that loading domestic refrigerators with water bottles can extend viable vaccine storage time during power loss situations \[[@pone.0235777.ref006],[@pone.0235777.ref007]\], but the precise relationship between thermal ballast quantity and its impact on temperature stability maintenance was not previously evaluated.

The viable vaccine storage times for each of the power loss trials are summarized in [Fig 6](#pone.0235777.g006){ref-type="fig"}. A Pearson correlation coefficient, r, was calculated as described in the Evaluation Methods section to assess the strength of the relationship between added thermal ballast and duration of viable vaccine storage time during an outage, for each tested unit. In the standalone refrigerator, r = 0.96, n = 68, p \< 0.0001, where n is the total number of observations (temperature sensors multiplied by number of trials), and p is the probability that the null hypothesis (added thermal ballast is not associated with an increase in viable storage time) is true. In the combination unit, r = 0.94, n = 40, p \< 0.0001. In both units, we observe a strong positive correlation between increased thermal ballast load and extended viable vaccine storage time during an outage event.

![Viable vaccine storage time during power loss trials conducted with varying thermal ballast load percentages.\
Round blue markers indicate the average viable vaccine storage time achieved during each trial, and error bars show the range of these measurements, corresponding to the within-trial variation among monitored vaccines.](pone.0235777.g006){#pone.0235777.g006}

A variety of factors, including refrigerator design, specific heat capacity, total thermal mass and placement of stored items, as well as ambient conditions, are likely to impact the length of time that a unit maintains appropriate vaccine storage temperatures in a power loss situation. Current guidelines stress the importance of continuous temperature monitoring during power outages, and implementation of emergency vaccine storage, handling, and transport procedures, including transfer of vaccines to an alternative storage facility, when warranted \[[@pone.0235777.ref005],[@pone.0235777.ref029]\]. However, the findings from our case study may be used to provide some baseline guidance on the performance of vaccine-loaded domestic units during outages of varying lengths. In the absence of a thermal ballast load, vaccines stored in the standalone unit exceeded viable storage temperatures in just over an hour, while vaccines stored in the combination unit lasted about two hours. In both units, a 17% ballast load extended viable vaccine storage time to more than 6 hours. Under similar operating conditions, a less-cumbersome 10% ballast load can be expected to provide about 4 hours of viable vaccine storage time during an outage.

Storage containers {#sec015}
------------------

The vaccine storage setup used throughout this study incorporated three types of storage containers (stainless steel instrument trays, open plastic shelf bins, and lidded plastic shelf bins). For each of the trials described in the preceding sections, we compared average temperature variations and viable storage time for vaccines stored in each of the different container types. We observed average variations in temperature response of 1 °C or less for vaccines stored in different tray types placed on the same refrigerator shelf ([S2 Appendix](#pone.0235777.s012){ref-type="supplementary-material"}). These variations presented no clear pattern across multiple trials and storage locations, suggesting no clear preference for one type of tray over another.

Numerous thermal engineering studies have evaluated and modeled positional temperature gradients, airflow, and convection patterns in domestic refrigerators \[[@pone.0235777.ref030]--[@pone.0235777.ref034]\]. It is no surprise that these dynamical variables are the primary drivers of stored product temperatures. As such, the strength of our conclusions about storage container types are limited by our study design, which kept each of the storage container types in fixed positions throughout the study. This configuration makes it impossible to separate the effects of storage location from any effects introduced by storage container type. Still, our results suggest that storage container characteristics (metal, plastic, open vs. closed) have minimal impact on the stability of stored vaccine temperatures.

Glass shelves vs. wire shelves {#sec016}
------------------------------

During the shelf material case study power outage tests, vaccines experienced too-warm temperature excursions within the first 1.5 hours of the outage period. Vaccines stored in trays placed on glass shelves were maintained within proper storage temperatures ($\overline{\Delta T} < 3{^\circ}C$) for on average, only 7 minutes longer than identically placed vaccines stored in trays kept on wire shelves. During the power outage recovery phase, vaccines recovered to their pre-outage setpoint temperatures within one to one and one-half hours once power was restored. In this case, vaccines in trays stored on glass shelves took, on average, only 3 minutes longer to recover to their pre-outage temperature setpoint, as compared to vaccines in trays kept on wire shelves.

During the 2 h repeated door opening test, approximately 60% of monitored vaccines stored in trays on wire shelves experienced a too-warm temperature excursion, while the remaining 40% were kept within appropriate storage temperatures ($\overline{\Delta T} < 3{^\circ}C$). By contrast, 100% of monitored vaccines stored in trays placed on glass shelves experienced a too-warm temperature excursion during the test ($\overline{\Delta T} > 3{^\circ}C$). Vaccines kept on wire shelves also recovered their pre-door opening setpoint temperature faster than vaccines stored on glass shelves (10 min vs. 23 min, on average).

In summary, this limited-scope test demonstrates that vaccines stored on wire shelves in a domestic refrigerator may experience slightly less temperature variability in response to repeated door opening events, as compared to overwise-identically stored vaccines placed on glass shelves. Because this test utilized an after-market glass shelf installation in a domestic refrigerator, these results should not be extrapolated to conclude that glass shelves are universally unfavorable for vaccine storage. Current or future vaccine storage products may exist in which the refrigeration design is successfully engineered to overcome reduced airflow and other challenges introduced by the inclusion of glass shelves.

Discussion {#sec017}
==========

The results described here support the practice of thermal ballast loading in domestic refrigerators used for vaccine storage. Thermal ballast usage is a practical and effective strategy for mitigating the negative impact of power outage events. Our results indicate that a thermal ballast load comprising 10% to 15% of the total refrigerator storage volume supports sustained vaccine storage temperature maintenance during power loss events for approximately 4 to 6 hours, which is within the range of typical U.S. power outage events. In addition, this window provides ample time for provider action and implementation of remediation plans during extended outages. If no thermal ballast load is used, vaccines stored in domestic refrigerators may experience temperature excursions after just 1.5 hours of sustained outage time.

Thermal ballast usage appears less effective at improving vaccine temperature stability during other types of adverse events. While a small thermal ballast load supports improved temperature stability during defrost cycle activation in a standalone unit, the combination unit maintains appropriate storage conditions during defrost cycle activation regardless of ballast usage.

Thermal ballast usage does not practically or reliably reduce the severity of temperature excursions caused by repeated door opening or accidental "door left open" events. Other tactics, including user training, alarm systems, and built-in engineering controls provide better results in limiting the incidence and negative impact of these types of events \[[@pone.0235777.ref011],[@pone.0235777.ref012],[@pone.0235777.ref014],[@pone.0235777.ref035]\]. Purpose-built vaccine refrigerators frequently include features like high-powered forced air convection, "air curtains", and deactivation of fans upon door opening, which limit vaccine exposure to ambient temperatures during routine door openings.

Our findings support the use of a moderate thermal ballast load, comprising 10% to 15% of the total refrigerator storage volume, in domestic units adapted for vaccine storage. [Fig 3](#pone.0235777.g003){ref-type="fig"} shows an example of a 10% ballast load in a standalone refrigerator. While even larger ballast loads could provide additional temperature stability in certain situations, we found loads greater than 17% to be impractical for vaccine storage applications, as too much of the usable refrigerator shelf space is lost to water bottles. Thermal ballast loads less than 10% are also likely to provide some extension in viable vaccine storage time during short power outages, as seen in [Fig 6](#pone.0235777.g006){ref-type="fig"}. A moderate ballast load provides an additional programmatic benefit by allowing the refrigerator manager to "cordon off" regions which have been shown to be inappropriate for vaccine storage, including the floor, the top shelf near the cooling vents in a combination unit, and the door of the unit \[[@pone.0235777.ref005]--[@pone.0235777.ref007]\].

For optimal temperature maintenance and control of stored vaccines, providers should use purpose-built, validated equipment whenever possible. However, in cases where a purpose-built vaccine refrigerator is cost-prohibitive or otherwise unavailable, domestic refrigerators may be used to safely store vaccine with additional training, oversight, and programmatic controls, such as thermal ballast loading.

The small sample size included in this study (two domestic refrigerator units) may limit the generalizability of the findings to some degree. However, because we found that thermal ballast loading is primarily useful as a power loss mitigation strategy, we expect this effect to extend to any refrigerator model. During a power outage, the rate of temperature rise inside a closed refrigerator is driven by the temperature difference between the interior and the environment, the rate of heat exchange through the refrigerator walls, and the heat capacity of the refrigerator's contents. Refrigerator model-specific characteristics like the insulation efficiency will influence the rate of heat exchange. However, the total heat capacity of a refrigerator's contents is a function of those contents' mass and specific heat capacity and is therefore model-independent. Pre-emptively loading a unit with added mass in the form of thermal ballast materials is a practical strategy for extending viable vaccine storage time during power loss events.

While our results indicate that a thermal ballast may be used to improve the temperature stability of vaccines stored in domestic refrigerators subject to certain adverse events, it may also be possible to negatively impact stored vaccines via excessive or improper ballast loading. While we did not identify disruptive temperature gradients created by the thermal ballast loading strategies employed in this study, careless placement of water bottles may unfavorably disrupt refrigerator airflow, exacerbating existing temperature gradients, or creating new ones. Furthermore, most users utilize a single temperature monitoring probe inside their vaccine refrigerators, and so may fail to detect these types of problems \[[@pone.0235777.ref036]\]. In addition, space constraints imposed by use of even moderate thermal ballast loads could lead to cramped vaccine storage conditions and complicate inventory management.

We emphasize the fact that our study was designed to maintain the same ratio of bottles per location for each ballast load percentage, and as a result, the water bottle placement shown in [Fig 3](#pone.0235777.g003){ref-type="fig"} may not represent the ideal setup for all units. In practice, we suggest filling the door of the unit first, then the floor of the unit, and finally the top shelf, since these areas are typically not suitable for vaccine storage. Users should take care to maintain airflow gaps between each refrigerator shelf and adjacent walls, to avoid creating dead pockets of warmer or colder air.

Finally, the suggestions in this publication are intended for application to domestic food storage refrigerators used to store vaccines and should not be extrapolated to purpose-built vaccine storage refrigerators. Purpose-built refrigerators are typically engineered to maintain appropriate vaccine storage temperatures throughout their entire usable storage volume during routine use. Most of these units undergo extensive temperature stability and performance validation by the manufacturer. Placement of a large thermal ballast load in a purpose-built unit may disrupt airflow distribution systems, resulting in unfavorable vaccine storage conditions. Users of purpose-built vaccine refrigerators should avoid following the thermal ballast loading practices described in this publication in the absence of manufacturer approval and model-specific guidance.

Conclusions {#sec018}
===========

We have quantified the effectiveness of thermal ballast loading using filled water bottles, a popular vaccine management strategy, in two domestic refrigerator units subjected to a variety of typical use conditions and adverse events. We observed a strong positive correlation between the thermal ballast load and viable vaccine storage time during power outages (standalone refrigerator, *r* = 0.96, *n* = 68, *p* \< 0.0001 and combination refrigerator/freezer, *r* = 0.94, *n* = 40, *p* \< 0.0001). Based on our findings, a moderate thermal ballast load, comprising ten to fifteen percent of the total refrigerator storage volume, supports maintenance of vaccine temperatures between 2 °C and 8 °C for 4 to 6 hours without power, and represents a practical strategy for mitigating temperature excursions risks in areas with frequent or protracted power outages. However, clinics utilizing backup generators and other power loss mitigation strategies have little to gain from this practice, as it does not reliably reduce the frequency or severity of temperature excursions caused by repeated door opening, accidental "door left open" events, or refrigerator defrost cycle activation. Furthermore, while thermal ballast loads greater than 17% provided additional viable vaccine storage time during power loss events, excessive thermal ballast usage restricts available vaccine storage space and could unfavorably disrupt refrigerator airflow. These findings support better, safer vaccine management by enabling providers to make informed, evidence-based decisions about thermal ballast usage.

Supporting information {#sec019}
======================

###### Photos of 25% and 8% thermal ballast load setups.

(DOCX)

###### 

Click here for additional data file.

###### Temperature response of seventeen monitored vaccine vials to defrost cycle activation in standalone refrigerator, with 0% thermal ballast load.

(PDF)

###### 

Click here for additional data file.

###### Combination refrigerator/freezer defrost cycle.

**$\overline{\Delta T}$** data shown in [Fig 4L](#pone.0235777.g004){ref-type="fig"}. "Pos error" and "neg error" show the extreme values recorded by all sensors, corresponding to the overall range.

(CSV)

###### 

Click here for additional data file.

###### Standalone refrigerator defrost cycle.

$\overline{\Delta T}$ data shown in [Fig 4R](#pone.0235777.g004){ref-type="fig"}. "Pos error" and "neg error" show the extreme values recorded by all sensors, corresponding to the overall range.

(CSV)

###### 

Click here for additional data file.

###### Combination refrigerator/freezer repeated door opening.

**$\overline{\Delta T}$** data shown in [Fig 5L](#pone.0235777.g005){ref-type="fig"}. "Pos error" and "neg error" show the extreme values recorded by all sensors, corresponding to the overall range.

(CSV)

###### 

Click here for additional data file.

###### Standalone refrigerator repeated door opening.

$\overline{\Delta T}$ data shown in [Fig 5R](#pone.0235777.g005){ref-type="fig"}. "Pos error" and "neg error" show the extreme values recorded by all sensors, corresponding to the overall range.

(CSV)

###### 

Click here for additional data file.

###### Combination refrigerator/freezer door left open.

Average duration of viable vaccine storage time with refrigerator door left open.

(CSV)

###### 

Click here for additional data file.

###### Standalone refrigerator door left open.

Average duration of viable vaccine storage time with refrigerator door left open.

(CSV)

###### 

Click here for additional data file.

###### Combination refrigerator/freezer power outage.

Average duration of viable vaccine storage time, as shown in [Fig 6L](#pone.0235777.g006){ref-type="fig"}. "Pos error" and "neg error" show the extreme values recorded by all sensors, corresponding to the overall range.

(CSV)

###### 

Click here for additional data file.

###### Standalone refrigerator power outage.

Average duration of viable vaccine storage time, as shown in [Fig 6R](#pone.0235777.g006){ref-type="fig"}. "Pos error" and "neg error" show the extreme values recorded by all sensors, corresponding to the overall range.

(CSV)

###### 

Click here for additional data file.

###### Comparison of temperature response by monitored vaccines stored in different types of trays.

(PDF)

###### 

Click here for additional data file.

###### Comparison of door opening temperature response by boxed and unboxed vaccine, at 0% and 25% thermal ballast loads.

(PDF)

###### 

Click here for additional data file.

The authors thank W. Wyatt Miller for technical assistance and fabrication support in this study.

10.1371/journal.pone.0235777.r001

Decision Letter 0

Vo

Dai-Viet N.

Academic Editor

© 2020 Dai-Viet N. Vo

2020

Dai-Viet N. Vo

This is an open access article distributed under the terms of the

Creative Commons Attribution License

, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

10 Mar 2020

PONE-D-20-03640

Effect of thermal ballast loading on temperature stability of domestic refrigerators used for vaccine storage

PLOS ONE

Dear Chojnacky,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE's publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

We would appreciate receiving your revised manuscript by Apr 24 2020 11:59PM. When you are ready to submit your revision, log on to <https://www.editorialmanager.com/pone/> and select the \'Submissions Needing Revision\' folder to locate your manuscript file.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter.

To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see: <http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols>

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). This letter should be uploaded as separate file and labeled \'Response to Reviewers\'.A marked-up copy of your manuscript that highlights changes made to the original version. This file should be uploaded as separate file and labeled \'Revised Manuscript with Track Changes\'.An unmarked version of your revised paper without tracked changes. This file should be uploaded as separate file and labeled \'Manuscript\'.

Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

We look forward to receiving your revised manuscript.

Kind regards,

Dai-Viet N. Vo, Ph.D.

Academic Editor

PLOS ONE

Journal Requirements:

1\. When submitting your revision, we need you to address these additional requirements.

 

Please ensure that your manuscript meets PLOS ONE\'s style requirements, including those for file naming. The PLOS ONE style templates can be found at

<http://www.journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf> and <http://www.journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf>

2\. We note that you have included the phrase "data not shown" in your manuscript. Unfortunately, this does not meet our data sharing requirements. PLOS does not permit references to inaccessible data. We require that authors provide all relevant data within the paper, Supporting Information files, or in an acceptable, public repository. Please add a citation to support this phrase or upload the data that corresponds with these findings to a stable repository (such as Figshare or Dryad) and provide and URLs, DOIs, or accession numbers that may be used to access these data. Or, if the data are not a core part of the research being presented in your study, we ask that you remove the phrase that refers to these data.

3\. Thank you for stating the following in the Competing Interests section:

\"I have read the journal\'s policy and the authors of this manuscript have the following competing interests:

This study was partially funded by an Interagency Agreement with Centers for Disease Control and Prevention (CDC) and the authors\' institution. As part of this agreement, MC provides technical guidance and research support to the CDC\'s Vaccines for Children program. Additionally, MC is a member of the NSF Joint Committee for Vaccine Storage, a consensus body responsible for creating and maintaining an ANSI standard for purpose-built vaccine storage equipment.\".

i\) Please confirm that this does not alter your adherence to all PLOS ONE policies on sharing data and materials, by including the following statement: \"This does not alter our adherence to  PLOS ONE policies on sharing data and materials." (as detailed online in our guide for authors <http://journals.plos.org/plosone/s/competing-interests>).  If there are restrictions on sharing of data and/or materials, please state these. Please note that we cannot proceed with consideration of your article until this information has been declared.

ii\) Please include your updated Competing Interests statement in your cover letter; we will change the online submission form on your behalf.

Please know it is PLOS ONE policy for corresponding authors to declare, on behalf of all authors, all potential competing interests for the purposes of transparency. PLOS defines a competing interest as anything that interferes with, or could reasonably be perceived as interfering with, the full and objective presentation, peer review, editorial decision-making, or publication of research or non-research articles submitted to one of the journals. Competing interests can be financial or non-financial, professional, or personal. Competing interests can arise in relationship to an organization or another person. Please follow this link to our website for more details on competing interests: <http://journals.plos.org/plosone/s/competing-interests>

4\. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: <http://journals.plos.org/plosone/s/supporting-information>.

\[Note: HTML markup is below. Please do not edit.\]

Reviewers\' comments:

Reviewer\'s Responses to Questions

**Comments to the Author**

1\. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer \#1: Yes

Reviewer \#2: Yes

\*\*\*\*\*\*\*\*\*\*

2\. Has the statistical analysis been performed appropriately and rigorously?

Reviewer \#1: Yes

Reviewer \#2: Yes

\*\*\*\*\*\*\*\*\*\*

3\. Have the authors made all data underlying the findings in their manuscript fully available?

The [PLOS Data policy](http://www.plosone.org/static/policies.action#sharing) requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data---e.g. participant privacy or use of data from a third party---those must be specified.

Reviewer \#1: Yes

Reviewer \#2: Yes

\*\*\*\*\*\*\*\*\*\*

4\. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer \#1: Yes

Reviewer \#2: Yes

\*\*\*\*\*\*\*\*\*\*

5\. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer \#1: The study entitled "Effect of thermal ballast loading on temperature stability of domestic refrigerators used for vaccine storage" by Michal Chojnacky et al. investigated the effect of thermal ballast loadings on the temperature stability of vaccine stored. In general, the manuscript is well written and organized, the language is excellent so it is easy to follow. The study is every interesting because the authors compared their results on two kinds of refrigerators involving combination and standalone. Moreover, many factors (defrost cycle, door open, power outage, storage containers, glass shelves vs. wire shelves) have been studied carefully. The results seem reliable with repeatable runs. The prospective of this study is promising and meaningful practically since it can be applied for isolated regions with a deficiency of cooling instruments or electricity. With these reasons, the reviewer is in favor of acceptance for publication after the manuscript is revised along with the following minor comments.

1\. The abstract of this manuscript seems quiet long and it should also be appropriately adhered according to the journal's guide, please check and shorten it. By the way, please check the table of reference contents.

2\. In the conclusion, I think that you should correct first two sentences because they don't come from your results. The conclusion should be a summary in the method, the most highlighted results, what the findings attained, etc. Please insert some numeric data to more consolidate the significance of this study.

3\. In Figs. 1-3, the name and position of the important materials (thermos ballast, vaccines, etc.) should be clarified. In addition, each type of vaccine is characterized by various conditions in critical temperature, light-exposure intensity, expiration date, etc. The same case is also applied for refrigerator. Please list these properties of those vaccines/refrigerator in your experiments to make sure that the further studies can be repeated to verify.

4\. In Fig. 6, there are increasing trends in viable storage time along with the percentage of ballast loading for both combination and standalone refrigerators. You are highly recommended to increase ballast loadings (e.g. 30%, 35%, 40%, or more) to track the result and detect the optimum ballast loadings.

Reviewer \#2: This work investigated the effect of thermal ballast loading on temperature stability of domestic refrigerators used for vaccine storage. The work is interesting with relevant application to the healthcare area. The discussion is well-written; nonetheless, certain parts especially the method section are unclear. Therefore, I would like to suggest a minor revision for this work.

1\) Introduction

a\) Please specify what kind of "certain" operating conditions.

"However, under certain operating conditions, domestic refrigerators can expose vaccines to temperatures outside the permissible storage range (8--13)."

2\) Refrigerator Selection

a\) How do the authors confirm the variation of shelve type do not affect the performance of refrigerator?

"The combination unit had three glass storage shelves inside its refrigerated compartment, and the standalone unit had four interior wire shelves."

b\) More specifications of refrigerators should be provided in Table 1. For instance, power consumption and etc.

c\) The model of refrigerators should be mentioned.

3\) Temperature Monitoring

a\) Any specific reason for using different number of thermocouples to monitor vaccines from different refrigerators?

"A total of twenty-nine Type T thermocouples were used to monitor vaccines in the two refrigerators, with twelve monitored vaccines in the combination unit, and seventeen monitored vaccines in the standalone unit."

b\) What is type A and type B temperature measurement systems? Are them refers to thermocouple? If yes, why use two different thermocouples? Confusing as they are not mentioned in the text.

4\) Thermal Ballast

a\) Are the large bottles and jugs also made from plastic? Different matters might have different heat conductivity.

b\) The formula of temperature stability should be included in the methodology section instead of results & discussion section.

5\) Test Protocol

a\) The set temperature should be specified in the text.

6\) Defrost cycle

a\) Transient profile of temperature should be provided to indicate any fluctuation over time.

b\) Please specify how many % of reduced temperature fluctuation inside bracket.

"...while the 8 % and 17 % ballast loads had little effect on the defrost cycle temperature response."

7\) Repeated Door Opening

a\) Redundant discussion. The way of evaluation should be declared in methodology rather than discussion.

"...with Δ \> 3 ° indicative of a potential excursion condition."

8\) Door Open for One Hour

a\) Similar to comment \#6a.

"As before, we examined the average temperature change of the monitored vaccine vials, Δ, to assess temperature stability during the trial. We define "viable vaccine storage time" as the duration of time in which vaccines were maintained within 3 °C of their initial, pre-door opening storage temperature (Δ \< 3 °)."

b\) I suggest the authors to provide the data as supplementary data.

"In this case, vaccines remained within a viable storage temperature range for an average of 22 minutes (data not shown)."

9\) Power Outage

a\) Similar to comment \#6a.

"To determine the impact of thermal ballast load on refrigerated vaccine temperature stability during a power outage event, we examined "viable vaccine storage time" during simulated outage tests performed at different ballast load percentages. As before, viable vaccine storage time is defined as the duration of time during which vaccine temperatures were maintained within 3 °C of their initial, pre-outage storage temperature (Δ \< 3 °). In this framework, a temperature rise of more than 3 °C is equivalent to the start of a temperature excursion, and the end of viable vaccine storage time."

b\) How the Pearson correlation coefficient being calculated? It should be included under the methodology section.

"Pearson correlation coefficient, r, was calculated to assess the strength of the relationship between added thermal ballast and duration of viable vaccine storage time during an outage, for each tested unit."

"...n is the total number of observations (temperature sensors multiplied by number of trials), and p is the probability that the null hypothesis (added thermal ballast is not associated with an increase in viable storage time) is true"

10\) Storage Containers

a\) How trivial it is? Please provide some numerical values or graph?

"Still, our results suggest that the selection of a storage container (metal, plastic, open vs. closed) is trivial in terms of its impact on maintaining stored vaccine temperatures."

11\) Glass Shelves vs. Wire Shelves

a\) The methodology of this section should be included under the methodology section.

b\) Please provide the specification of the glass sheet that used to produce glass shelves.

12\) Discussions

a\) It is well-written. Well done.

13\) Conclusion

a\) The conclusion should be more specific in accordance to the current findings. Please support your recommendation/suggestion with facts like data.

\*\*\*\*\*\*\*\*\*\*

6\. PLOS authors have the option to publish the peer review history of their article ([what does this mean?](https://journals.plos.org/plosone/s/editorial-and-peer-review-process#loc-peer-review-history)). If published, this will include your full peer review and any attached files.

If you choose "no", your identity will remain anonymous but your review may still be made public.

**Do you want your identity to be public for this peer review?** For information about this choice, including consent withdrawal, please see our [Privacy Policy](https://www.plos.org/privacy-policy).

Reviewer \#1: No

Reviewer \#2: No

\[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link \"View Attachments\". If this link does not appear, there are no attachment files to be viewed.\]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, <https://pacev2.apexcovantage.com/>. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at <figures@plos.org>. Please note that Supporting Information files do not need this step.

10.1371/journal.pone.0235777.r002

Author response to Decision Letter 0

30 May 2020

Response to Reviewers letter has been uploaded as a separate file.

###### 

Submitted filename: Response to Reviewers.docx

###### 

Click here for additional data file.

10.1371/journal.pone.0235777.r003

Decision Letter 1

Vo

Dai-Viet N.

Academic Editor

© 2020 Dai-Viet N. Vo

2020

Dai-Viet N. Vo

This is an open access article distributed under the terms of the

Creative Commons Attribution License

, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

23 Jun 2020

Effect of thermal ballast loading on temperature stability of domestic refrigerators used for vaccine storage

PONE-D-20-03640R1

Dear Dr. Chojnacky,

We're pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you'll receive an e-mail detailing the required amendments. When these have been addressed, you'll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at <http://www.editorialmanager.com/pone/>, click the \'Update My Information\' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at <authorbilling@plos.org>.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they'll be preparing press materials, please inform our press team as soon as possible \-- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact <onepress@plos.org>.

Kind regards,

Dai-Viet N. Vo, Ph.D.

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

The paper has been properly revised. Thus, it could be considered for publication.

Reviewers\' comments:

Reviewer\'s Responses to Questions

**Comments to the Author**

1\. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the "Comments to the Author" section, enter your conflict of interest statement in the "Confidential to Editor" section, and submit your \"Accept\" recommendation.

Reviewer \#1: All comments have been addressed

Reviewer \#2: All comments have been addressed

\*\*\*\*\*\*\*\*\*\*

2\. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer \#1: Yes

Reviewer \#2: Yes

\*\*\*\*\*\*\*\*\*\*

3\. Has the statistical analysis been performed appropriately and rigorously?

Reviewer \#1: Yes

Reviewer \#2: Yes

\*\*\*\*\*\*\*\*\*\*

4\. Have the authors made all data underlying the findings in their manuscript fully available?

The [PLOS Data policy](http://www.plosone.org/static/policies.action#sharing) requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data---e.g. participant privacy or use of data from a third party---those must be specified.

Reviewer \#1: Yes

Reviewer \#2: Yes

\*\*\*\*\*\*\*\*\*\*

5\. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer \#1: Yes

Reviewer \#2: Yes

\*\*\*\*\*\*\*\*\*\*

6\. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer \#1: I suggest to accept this manuscript for publication because the authors have responded all of the comments correctly and appropriately.

Reviewer \#2: The manuscript reported an interesting work on \"Effect of thermal ballast loading on temperature stability of domestic refrigerators used for vaccine storage\". Well done for the improvement. In my opinion, the manuscript is ready for publication. All the comments have been properly addressed.

\*\*\*\*\*\*\*\*\*\*

7\. PLOS authors have the option to publish the peer review history of their article ([what does this mean?](https://journals.plos.org/plosone/s/editorial-and-peer-review-process#loc-peer-review-history)). If published, this will include your full peer review and any attached files.

If you choose "no", your identity will remain anonymous but your review may still be made public.

**Do you want your identity to be public for this peer review?** For information about this choice, including consent withdrawal, please see our [Privacy Policy](https://www.plos.org/privacy-policy).

Reviewer \#1: No

Reviewer \#2: No

10.1371/journal.pone.0235777.r004

Acceptance letter

Vo

Dai-Viet N.

Academic Editor

© 2020 Dai-Viet N. Vo

2020

Dai-Viet N. Vo

This is an open access article distributed under the terms of the

Creative Commons Attribution License

, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

26 Jun 2020

PONE-D-20-03640R1

Effect of thermal ballast loading on temperature stability of domestic refrigerators used for vaccine storage

Dear Dr. Chojnacky:

I\'m pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they\'ll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact <onepress@plos.org>.

If we can help with anything else, please email us at <plosone@plos.org>.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Dai-Viet N. Vo

Academic Editor

PLOS ONE

[^1]: **Competing Interests:**I have read the journal's policy and the authors of this manuscript have the following competing interests: This study was partially funded by an Interagency Agreement with Centers for Disease Control and Prevention (CDC) and the authors' institution. As part of this agreement, MC provides technical guidance and research support to the CDC's Vaccines for Children program. Additionally, MC is a member of the NSF Joint Committee for Vaccine Storage, a consensus body responsible for creating and maintaining an ANSI standard for purpose-built vaccine storage equipment. This does not alter our adherence to PLOS ONE policies on sharing data and materials.

[^2]: Current address: Independent Researcher, Bethesda, Maryland, United States of America
