To date, traditional methods for assessing wine quality and the degree of smoke taint have involved the use of chromatographic techniques for the identification of aroma volatiles and trained panels [4,39–41]. However, there are several drawbacks to these techniques, as they can be time-consuming in terms of sample preparation, as well as training sensory experts, which is expensive, as chromatographic techniques require costly reagents and training, and maintaining trained panels can also be expensive, as well as being destructive in their forms of assessment. Furthermore, the results from sensory evaluations using human panels can be affected by physiological and psychological issues within the individuals, such as fatigue and decreased sensitivity to samples due to prolonged exposure [4,39–41]. Thus, the use of low-cost e-noses may offer a more costeffective and accurate form of quality and smoke taint assessments. Previous research by Fuentes et al. [18] developed ANN models using readings from a low-cost e-nose as inputs to accurately predict the levels of volatile phenols and their glycoconjugates in grapes and wine, as well as a model predicting 12 consumer sensory responses towards the wine. The models developed in their study may offer grape growers a rapid and cost-effective technique for assessing smoke-tainted wines; however, the models have yet to be tested on wines that have undergone smoke taint amelioration techniques to assess the accuracy of the models following treatment with activated carbon.

The ANN model developed in this study showed high accuracy in classifying the e-nose readings according to the different smoke taint amelioration techniques applied. It may offer winemakers a rapid, cost-effective tool to assess the effectiveness of smoke taint amelioration by activated carbon with/without the addition of a cleaving enzyme. Furthermore, as this method is nondestructive, repeated measurements can be made to assess wine samples as they age over time. In addition to this, when the STCE enzyme enose readings were assessed using the model, most of the readings were classified as either NSCO or NSAC, illustrating that the enzyme treatment together with activated carbon was effective in ameliorating smoke taint. Previous research by Fudge et al. [15] found that treatment with activated carbon alone was effective in removing smoke-derived volatile phenols; however, it did not remove their glycoconjugates. Therefore, the hydrolysis of these glycoconjugates over time may result in a resurgence of smoke aromas. The work by Herderich and Krstic [42] found that the treatment of smoke-affected Pinot Noir wine with a glucosidase enzyme reduced the concentrations of six glycosides by approximately 50%. Thus, combining the use of cleaving enzymes with activated carbon may enhance smoke taint amelioration by removing a significant amount of glycoconjugates and, therefore, delay the resurgence of smoke aromas as the wine ages. Further research is required to assess the exact levels of the glycoconjugates in the final wine samples.

### **5. Conclusions**

The use of an electronic nose coupled with machine learning modeling can offer winemakers a cost-effective and rapid technique for assessing the effectiveness of smoke taint amelioration treatment by activated carbon with/without the addition of a cleaving enzyme without requiring the use of a trained sensory panel. Related to the latter, further advantages are in the repeatability of the integrated hardware (e-nose) and software (AI), which allows the monitoring of multiple batches at the same time by integrating local analytical microprocessors (e.g., Jetson ® from NVIDIA®). Furthermore, the use of a cleaving enzyme can enhance smoke taint amelioration by removing a significant proportion of glycoconjugates, thereby delaying the resurgence of smoke aromas and maintaining the value of the wine for a longer period. The integrated system proposed could offer winemakers a near-real-time system to assess the effect of treatments on microbrewing for decision-making purposes to manage the smoke taint in wines.

**Author Contributions:** Conceptualization, V.S., S.F. and C.G.V.; methodology, V.S.; formal analysis, V.S. and C.G.V.; investigation, V.S. and C.G.V.; software, C.G.V. and S.F.; resources, S.F; data curation, V.S and C.G.V.; writing—original draft preparation, V.S.; writing—review and editing, C.G.V., A.P., D.D.T., V.S. and S.F.; visualization, V.S.; supervision, S.F., A.P. and D.D.T.; validation, C.G.V., V.S. and S.F.; project administration, S.F. and funding acquisition, S.F. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was supported through the Australian Government Research Training Program Scholarship.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** The data and intellectual property belong to The University of Melbourne; any sharing needs to be evaluated and approved by the university.

**Acknowledgments:** The authors greatly acknowledge Brown Brothers Winemakers and Wendy Cameron for organizing and supplying the wine samples and Enologica Vason for supplying the compounds for this research. The authors would like to acknowledge Ranjith R. Unnithan and Bryce Widdicombe from the School of Engineering, Department of Electrical and Electronic Engineering of The University of Melbourne for their collaboration in the electronic nose development.

**Conflicts of Interest:** The authors declare no conflict of interest.

### **References**

### *Article* **Predicting Alcohol Concentration during Beer Fermentation Using Ultrasonic Measurements and Machine Learning**

**Alexander Bowler 1 , Josep Escrig 2 , Michael Pound 3 and Nicholas Watson 1,\***

**Abstract:** Beer fermentation is typically monitored by periodic sampling and off-line analysis. Inline sensors would remove the need for time-consuming manual operation and provide real-time evaluation of the fermenting media. This work uses a low-cost ultrasonic sensor combined with machine learning to predict the alcohol concentration during beer fermentation. The highest accuracy model (R2 = 0.952, mean absolute error (MAE) = 0.265, mean squared error (MSE) = 0.136) used a transmission-based ultrasonic sensing technique along with the measured temperature. However, the second most accurate model (R2 = 0.948, MAE = 0.283, MSE = 0.146) used a reflection-based technique without the temperature. Both the reflection-based technique and the omission of the temperature data are novel to this research and demonstrate the potential for a non-invasive sensor to monitor beer fermentation.

**Keywords:** machine learning; ultrasonic measurements; long short-term memory; industrial digital technologies

### **1. Introduction**

During beer fermentation, yeast metabolism produces ethanol and carbon dioxide from a sugar-water mixture called wort [1,2]. The fermentation is conventionally monitored through off-line wort density measurements until a predetermined ethanol concentration is reached [3], after which the process is continued for a predefined time for development of flavour compounds [4]. This requires manual sampling, takes time, and wastes resources by disposing of the measured sample. In-line measurement techniques directly measure the process material and on-line methods use bypasses to automatically collect, analyse, and return samples to the process [5]. By providing real-time, automatic alcohol concentration measurements, in-line and on-line techniques would ensure product quality through early detection of anomalous batches, allow effective scheduling of production equipment by predicting fermentation endpoint, and reduce the burden of manual sampling by operators. Furthermore, real-time data is key to the Fourth Industrial Revolution, which will implement industrial digital technologies such as the Internet of Things, cloud computing, and machine learning (ML) to integrate entire processes, automatically make decisions, and improve manufacturing productivity, efficiency, and sustainability [6].

Several in-line and on-line methods to monitor alcoholic fermentation have been investigated, including in-situ transflectance near-infrared spectroscopy [7,8], and Raman spectroscopy probes [9]; automated flow-through mid-infrared spectroscopy [10], Fourier transform infrared spectroscopy [11], and piezoelectric MEMS resonators [12]; non-invasive Raman spectroscopy through transparent vessel walls [13]; and CO2 emission monitoring [14].  *Fermentation* **2021**, *7*, 34. https://doi.org/10.3390/ fermentation7010034

Received: 16 February 2021 Accepted: 2 March 2021 Published: 4 March 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

cost and have previously been used to study fermentation, including as in-line methods on circulation lines [15], in-situ in tanks [16], and using non-invasive, through-transmission of the fermenting media [17,18]. US monitoring techniques use high frequency (>1 MHz) and low power (<1 Wcm−2 ) pressure waves to characterise material properties whilst causing no alterations to the material in which they propagate [19]. However, US properties vary with temperature and the presence of gas bubbles causes attenuation of the sound wave [20]. Previous in-line, on-line, and off-line studies to monitor fermentation using US measurements have developed empirical or semi-empirical models from the speed of sound or acoustic impedance to determine alcohol content [16]. These methods require extensive calibration procedures to compensate for the effects of temperature, dissolved CO2 [16,18,21], and yeast cell concentration [18]. Supervised ML uses data to train predictive algorithms for classification or regression problems. Through ML, compensation procedures are not required as the complexities caused by varying process parameters imbedded in the sensor data can be unravelled. Furthermore, procedures for accurate determination of the speed of sound are not necessary [15,16,22].

This work presents three novel contributions to US monitoring of alcoholic fermentations: Firstly, ML is used to predict alcohol concentration during lab-scale beer fermentations from US measurements. Secondly, although an in-situ sensor probe is used, the potential for non-invasive monitoring of fermentation is investigated by only using the US wave reflected from the interface between the probe and the wort. This technique is similar to previous work by our group [23–26]. Implementation of this technique would provide inline, non-invasive process monitoring without the need for circulation or bypass lines. This method would also not require transmission through the total vessel contents, which would be impossible at industrial scale. Therefore, this technique could be inexpensively fitted to the outside of existing vessels. Finally, exclusion of the temperature as a feature in the ML models is evaluated. Effective monitoring without the need for an invasive temperature sensor would further reduce the cost and complexity of industrial implementation.

### **2. Materials and Methods**

The fermentation was conducted in a 30 L cylindrical plastic vessel (Figure 1). A lid sealed the vessel to protect the wort from contamination. The lid contained an air lock to release the CO2 produced during fermentation. A belt heater increased the temperature of the wort to facilitate fermentation. The wort was prepared in the vessel by dissolving and mixing 1.5 kg of malt (Coopers Real Ale, UK) and 1 kg of sugar (brewing sugar, the Home Brew Shop, UK) in 22 L of water. Once the ingredients were mixed, a US probe was installed, consisting of a US transducer (Sonatest, 2 MHz central frequency, UK) and a temperature sensor (RTD, PT1000, UK). The US transducer was connected to a Lecouer Electronique US Box (France) that excited the transducer and digitised the received US signal. The temperature sensor was connected to a Pico electronic box (PT-104 Data Logger, UK). The two electronic boxes were connected to a laptop that controlled the data acquisition. Coupling gel was applied between the US transducer and the probe, and a spring was used to maintain the contact pressure. A Tilt hydrometer was installed to provide real-time density measurements. The real-time density measurements were required as the ground truth data of the wort alcohol concentration to train the ML models. This device was a small cylinder that floats in the liquid with its centre of gravity different from its centre of buoyancy. This causes an inclination of the device that is dependent on the specific gravity of the fermenting media. The inclination of the hydrometer was measured by a self-contained accelerometer and was transmitted by radio to a smartphone located outside of the vessel. A calibration procedure related the inclination to the specific gravity. It should be noted hydrometers are not suitable for in-line monitoring of industrial fermentations. Firstly, the balance of the device can be easily distorted by foam or solids floating on the surface, or by bubbles produced during fermentation. Secondly, as the hydrometer floats on the wort surface, it would need manual removal at the end of each fermentation batch. The most accurate method of specific gravity measurement is to extract

samples and use a portable density meter. However, this would require manual sample withdrawal at least every 2 h and would decrease the volume of liquid in the vessel, affecting the fermentation process. Furthermore, this would only produce sparse ground truth measurements of the density to train the ML models. lower section of the vessel. This produced temperature variations from around 20 to 30 °C. However, this temperature variation is beneficial to our ML evaluation as each model must be able to generalise across a wide range of process temperatures.

floats on the wort surface, it would need manual removal at the end of each fermentation batch. The most accurate method of specific gravity measurement is to extract samples and use a portable density meter. However, this would require manual sample withdrawal at least every 2 h and would decrease the volume of liquid in the vessel, affecting the fermentation process. Furthermore, this would only produce sparse ground truth

The yeast (Coopers Real Ale, UK) was distributed on the surface and the vessel sealed.

The mixture was left for 4 to 7 days while the fermentation occurred. After this time, the fermentation equipment was cleaned and a new batch was prepared. In total, 13 batches were completed. During fermentation, data was collected from the three different sensors: the US sensor, the temperature sensor, and the hydrometer. The time of each measurement

tory changed during this time. Furthermore, the belt heater was only in contact with the

*Fermentation* **2021**, *7*, 34 3 of 13

measurements of the density to train the ML models.

**Figure 1.** Experimental apparatus and measured US wave reflections. **Figure 1.** Experimental apparatus and measured US wave reflections.

Sets of US and temperature data were collected periodically. Each of the sets consisted of 36 US waves and 36 temperature readings. For the US signal, 7000 sampling points were collected at 80 MHz sampling frequency. The time between each wave acquisition was 0.55 s. Between each set of data collection, 200 s elapsed. As depicted in Figure 1, the US transducers emitted sound waves which travelled along a PMMA buffer. At the interface between the buffer material and the wort, part of the sound wave is reflected back to the transducer (the 1st reflection). The rest of the sound wave continues through The yeast (Coopers Real Ale, UK) was distributed on the surface and the vessel sealed. The mixture was left for 4 to 7 days while the fermentation occurred. After this time, the fermentation equipment was cleaned and a new batch was prepared. In total, 13 batches were completed. During fermentation, data was collected from the three different sensors: the US sensor, the temperature sensor, and the hydrometer. The time of each measurement was also recorded. The fermentation batches were conducted over a period of approximately 3 months. This meant that the environmental and water temperature in the laboratory changed during this time. Furthermore, the belt heater was only in contact with the lower section of the vessel. This produced temperature variations from around 20 to 30 ◦C. However, this temperature variation is beneficial to our ML evaluation as each model must be able to generalise across a wide range of process temperatures.

the wort, reflects at the opposite probe wall, and travels back to the transducer to be recorded (the 2nd reflection). An example of the signal recorded by the transducer is presented in Figure 2a. Close-ups of each reflection are presented in Figure 2b,c. The first section of the waveform (sample points < 500) is reflected back to the transducer before contacting the buffer material and wort interface and therefore contains no useful information about the process. The 1st reflection is identified between sample point 900 and 1500, and the 2nd reflection between 6000 and 6500, as shown in Figure 2. Sets of US and temperature data were collected periodically. Each of the sets consisted of 36 US waves and 36 temperature readings. For the US signal, 7000 sampling points were collected at 80 MHz sampling frequency. The time between each wave acquisition was 0.55 s. Between each set of data collection, 200 s elapsed. As depicted in Figure 1, the US transducers emitted sound waves which travelled along a PMMA buffer. At the interface between the buffer material and the wort, part of the sound wave is reflected back to the transducer (the 1st reflection). The rest of the sound wave continues through the wort, reflects at the opposite probe wall, and travels back to the transducer to be recorded (the 2nd reflection). An example of the signal recorded by the transducer is presented in Figure 2a. Close-ups of each reflection are presented in Figure 2b,c. The first section of the waveform (sample points < 500) is reflected back to the transducer before contacting the buffer material and wort interface and therefore contains no useful information about the process. The 1st reflection is identified between sample point 900 and 1500, and the 2nd reflection between 6000 and 6500, as shown in Figure 2.

**Figure 2.** Example ultrasonic waveform obtained: (**a**) The 1st reflection is located around sample point 1000, the 2nd reflection is located around sample point 6000; (**b**) a close-up of the 1st reflection; (**c**) a close-up of the 2nd reflection. **Figure 2.** Example ultrasonic waveform obtained: (**a**) The 1st reflection is located around sample point 1000, the 2nd reflection is located around sample point 6000; (**b**) a close-up of the 1st reflection; (**c**) a close-up of the 2nd reflection.

#### *2.1. Volume of Alcohol Calculation 2.1. Volume of Alcohol Calculation*

The volume of alcohol (%) can be calculated from the specific gravity of the fermenting media using Equation (1) [27]. The volume of alcohol (%) can be calculated from the specific gravity of the fermenting media using Equation (1) [27].

$$\text{ABV} = (\text{SG}\_{\text{in}} - \text{SG}) \times 131.25 \,\text{J} \tag{1}$$

before the yeast was added, and SG is the current specific gravity of the fermenting liquid. The multiplier of this equation is based on the stoichiometric relationship of the fermentation reaction, where the decreasing density is due to CO2 production and escape through the air lock [28]. where ABV is the alcohol by volume (%), SGin is the starting specific gravity of the liquid before the yeast was added, and SG is the current specific gravity of the fermenting liquid. The multiplier of this equation is based on the stoichiometric relationship of the fermentation reaction, where the decreasing density is due to CO2 production and escape through the air lock [28].

where ABV is the alcohol by volume (%), SGin is the starting specific gravity of the liquid

### *2.2. Ultrasonic Wave Features 2.2. Ultrasonic Wave Features*

The following features were calculated from the obtained US waveform to use in the ML models. These are common features extracted from US waveforms [29]. The theory behind the selection of each feature is presented in their respective sections. Different combinations of these features were tested during ML model optimisation. The optimal feature combinations are presented in Table 1, Section 3.1. The following features were calculated from the obtained US waveform to use in the ML models. These are common features extracted from US waveforms [29]. The theory behind the selection of each feature is presented in their respective sections. Different combinations of these features were tested during ML model optimisation. The optimal feature combinations are presented in Table 1, Section 3.1.

#### 2.2.1. Energy 2.2.1. Energy

The waveform energy is a measure of the size of the waveform received by the transducer. For the 1st reflection, this is a measure of the proportion of the sound wave reflected from the interface between the buffer material and the wort. This is dependent on the change in acoustic impedance between these two materials [30]. Monitoring the waveform energy of the 2nd reflection offers additional information on the level of sound wave attenuation in the wort. This is caused by viscous losses in the media and scattering due to heterogeneities such as bubbles and yeast cells [30]. The waveform energy is a measure of the size of the waveform received by the transducer. For the 1st reflection, this is a measure of the proportion of the sound wave reflected from the interface between the buffer material and the wort. This is dependent on the change in acoustic impedance between these two materials [30]. Monitoring the waveform energy of the 2nd reflection offers additional information on the level of sound wave attenuation in the wort. This is caused by viscous losses in the media and scattering due to heterogeneities such as bubbles and yeast cells [30].

$$E = \sum\_{i=start}^{i=end} A\_i^2 \,\tag{2}$$

where *E* is the waveform energy, *Ai* is the waveform amplitude at sample point *i*, and *start* and *end* denote the range of samples points for the reflection of interest [29].

**Table 1.** Results for the long short-term memory neural network (LSTM) models. MAE: mean absolute error; MSE: mean squared error; ABV: alcohol by volume. The regression metrics for the models evaluated on the test set are highlighted in bold at the bottom of the table.

2.2.2. Peak-to-Peak Amplitude, Maximum Amplitude, and Minimum Amplitude

The peak-to-peak amplitude, maximum amplitude, and minimum amplitude provide additional information as to how the energy is distributed in the waveform. Changes in wort composition or temperature may affect how the sound wave travels and reflects from boundaries, presenting differences in the shape of the received waveform. These three features were calculated for both the 1st and 2nd reflections.

$$\text{PPA} = \max(\text{A}\_{\text{startened}}) - \min(\text{A}\_{\text{starterend}}),\tag{3}$$

$$\mathbf{A}\_{\text{max}} = \max(\mathbf{A}\_{\text{startend}}),\tag{4}$$

$$\mathbf{A}\_{\text{min}} = \min(\mathbf{A}\_{\text{starttend}}),\tag{5}$$

where PPA is the peak-to-peak amplitude, Amax is the maximum amplitude, and Amin is the minimum amplitude.

### 2.2.3. Energy Standard Deviation

A total of 36 US waves were collected during each acquisition block. Phenomena in the process—e.g., the presence of bubbles at different times during fermentation—may cause fluctuations in the energy of the received waveforms. Therefore, the standard deviation of the energy in a block of acquired waveforms was investigated as a feature. The standard deviation of the energy was calculated for both the 1st and 2nd reflections.

$$STD = \sqrt{\frac{1}{W} \sum\_{i=1}^{i=W} \left(E\_i - \overline{E}\right)^2} \tag{6}$$

where *STD* is the standard deviation, *W* is the number of waveforms collected in the block, *i* is an individual waveform, and *E* is the mean waveform energy in the block.

### 2.2.4. Time of Flight

The time of flight was calculated using a thresholding method, i.e., the waveform sample point where the second reflection amplitude rises above the signal noise. This is a measure of the speed of sound in the wort that is dependent on its density and compressibility [20].

### 2.2.5. Feature Gradients

A one-sided, backwards moving mean was applied to obtain lagged feature representations over the previous 5 h. For the artificial neural networks (ANNs), this allows the use of past process information. For the long short-term memory neural networks (LSTMs), this allows for a way of storing past process information in some features, reducing the burden on the LSTM units to remember all feature trajectories.

### *2.3. Machine Learning*

The ground truth data for the percentage volume of alcohol during fermentation was calculated from the portable density meter and hydrometer measurements. In total, 13 fermentation batches were monitored. The final two batches were selected as the test set to provide an unbiased assessment of the experimental methodology used. In an industrial setting, the final ML models would be deployed after collecting the training set runs. The remaining 11 batches were used in a 5-fold cross-validation procedure to optimize the ML models' hyperparameters. Long short-term memory neural networks (LSTMs) are able to retain information from previous time-steps in a sequence. LSTMs are a type of recurrent neural network that reduces the likelihood of vanishing or exploding gradients by using gate units. This enables their use over much longer sequences [31]. To evaluate the utility of using LSTMs to predict alcohol concentration, they were compared with artificial neural networks (ANNs) which are unable to store past process information. ANNs combine input features to produce new features which can approximate the relationship with the target variable given enough neurons in the hidden layer [32,33].

For the LSTMs, zero-padding was applied to the US features to make every fermentation batch sequence an equal length. A masking layer specified that the LSTM units ignore this padding. Each sequence consisted of 4646 timesteps. All timesteps for each batch were used as a single sequence rather than being split into multiple sequences of shorter length. While long LSTM sequences (250–500 timesteps) are prone to produce vanishing gradients when predicting a single output, this is not a problem when predicting an output at every timestep as used in this task [34].

For the ANNs, a single hidden layer and the Adam optimisation algorithm was used. Cross-validation determined the optimal batch size, number of neurons in the hidden layer, learning rate, drop-out rate, L2 regularisation penalty, and number of epochs for training. For the LSTMs, the Adam optimisation algorithm was used and the cross-validation procedure determined the optimal batch size, number of LSTM units, learning rate, dropout rate, L2 regularisation penalty, gradient norm clipping value, and number of epochs. After cross-validation, the set of hyperparameters which resulted in the lowest average validation error were used to train a final model using all of the training set. The networks were trained using TensorFlow 2.3.0. The coefficient of determination (R2 ), mean squared error (MSE) and mean absolute error (MAE) were used as performance metrics to evaluate the ML models. Multiple metrics produce a comprehensive assessment of a model's ability to fit to the test set and improve comparison between models.

### **3. Results**

Figure 3 displays selected features from all the fermentation batches. It is shown that the energy of 1st reflection (Figure 3a), energy of the 2nd reflection (Figure 3b), and the

time of flight of the sound wave through the wort (Figure 3c) start at different values for each batch. There are several explanations for this. Firstly, as presented in Figure 3d, the process temperature is not the same at the start of each batch. As the speed of sound is highly dependent on temperature, the US properties begin from different magnitudes. Secondly, the US probe required manual removal and repositioning when disposing each batch after fermentation. This disturbed the spring maintaining the contact pressure of the US transducer, which affects the sound energy transferred through the materials from the sensor. due to a disturbance of the sensor positioning after Batch 10. These batches were kept in order to provide an unbiased assessment of the experimental methodology used. In an industrial setting because the test set data (batches 12 and 13) would not be available to analyse prior to the ML model training. The energy of the 2nd reflection is diminished compared with the 1st reflection at the beginning of the fermentation until approximately Day 3, as presented in Figure 4. A similar result was found in [17] and is due to the fermentation being most vigorous at the start of the process. This causes more CO2 bubbles to be produced and therefore greater attenuation of the sound wave travelling through the wort.

**Figure 3.** US waveform features for all fermentation batches: (**a**) The energy of the 1st reflection; (**b**) the energy of the 2nd reflection; (**c**) the time of flight for the 2nd reflection; (**d**) the process temperature. **Figure 3.** US waveform features for all fermentation batches: (**a**) The energy of the 1st reflection; (**b**) the energy of the 2nd reflection; (**c**) the time of flight for the 2nd reflection; (**d**) the process temperature.

The trajectories of the waveform features are also not smooth. Again, this is partly due to the oscillating process temperature. In addition, bubbles of CO2 produced during the fermentation were observed to attach to the surface of the probe material, which would cause scattering and reflection of the sound wave. During the fermentation, as further CO2 bubbles were produced, the new bubbles would replace the previous ones on the surface. This is likely to cause fluctuations in the waveform energy transferring through the interface between the probe and the wort.

The energy of the 1st reflection increases throughout the fermentation (Figure 3a). The energy of the 1st reflection is proportional to the change in acoustic impedance at the buffer-wort interface, with the acoustic impedance being a product of the material density and speed of sound [20]. As the density of the wort decreases during fermentation, the speed of sound also decreases as found in [17,18,35]. As the solid buffer material has a

greater density and speed of sound than the starting wort, the proportion of sound wave reflected at the buffer-wort material increases throughout the fermentation. However, the time of flight (Figure 3c), the inverse of the speed of sound, shows no general trend. This contrasts with the results obtained in [17,18,35], which suggests that it should increase. This is likely due to the changing process temperature masking an increasing time of flight. The results in [17,18,35] were all obtained at a constant temperature. The reduced time of flight for the last three batches (Batches 11, 12, and 13 in Figure 3c) is most likely due to a disturbance of the sensor positioning after Batch 10. These batches were kept in order to provide an unbiased assessment of the experimental methodology used. In an industrial setting because the test set data (batches 12 and 13) would not be available to analyse prior to the ML model training. The energy of the 2nd reflection is diminished compared with the 1st reflection at the beginning of the fermentation until approximately Day 3, as presented in Figure 4. A similar result was found in [17] and is due to the fermentation being most vigorous at the start of the process. This causes more CO2 bubbles to be produced and therefore greater attenuation of the sound wave travelling through the wort. (**c**) (**d**) **Figure 3.** US waveform features for all fermentation batches: (**a**) The energy of the 1st reflection; (**b**) the energy of the 2nd reflection; (**c**) the time of flight for the 2nd reflection; (**d**) the process temperature.

(**a**) (**b**)

*Fermentation* **2021**, *7*, 34 7 of 13

due to a disturbance of the sensor positioning after Batch 10. These batches were kept in order to provide an unbiased assessment of the experimental methodology used. In an industrial setting because the test set data (batches 12 and 13) would not be available to analyse prior to the ML model training. The energy of the 2nd reflection is diminished compared with the 1st reflection at the beginning of the fermentation until approximately Day 3, as presented in Figure 4. A similar result was found in [17] and is due to the fermentation being most vigorous at the start of the process. This causes more CO2 bubbles to be produced

and therefore greater attenuation of the sound wave travelling through the wort.

**Figure 4.** A comparison between the waveform energy of the 1st and 2nd reflections for batches 1 and 2. The energy of the 2nd reflection is diminished compared to the 1st until approximately Day 3. This is due to sound wave attenuation because of CO2 bubbles being produced.

### *3.1. Machine Learning*

The ANN model with the highest accuracy only achieved an R2 of 0.398 (MAE = 1.010% ABV, MSE = 1.942% ABV). As such, only results from the LSTM models are included in Table 1. This shows that the gradients of the features, as provided to the ANNs, is insufficient, and the enhanced memory of the feature history provided by the LSTM units is required for this process. The results of four final LSTM models are presented in Table 1, which either use the 1st reflection or both reflections, and either use the temperature as a feature or not. The optimal features, optimal hyperparameters, and performance metrics are included. The most accurate LSTM model (Model 1) used features from both the 1st and 2nd reflections and the process temperature. Interestingly, the second most accurate model (Model 4) only used features from the 1st reflection, excluding the process temperature. The third most accurate model (Model 2) used features from the 1st and 2nd reflections without the process temperature. Finally, the least accurate model (Model 3) combined features from the 1st reflection and the process temperature. Graphical representations of these predictions are shown in Figure 5a–h.

**Figure 5.** Predicted alcohol by volume percentage from the final LSTM models for the two test set batches (batches 12 and 13): (**a**) Model 1, Batch 12; (**b**) Model 1, Batch 13; (**c**) Model 2, Batch 12; (**d**) Model 2, Batch 13; (**e**) Model 3, Batch 12; (**f**) Model 3, Batch 13; (**g**) Model 4, Batch 12; (**h**) Model 4, Batch 13. **Figure 5.** Predicted alcohol by volume percentage from the final LSTM models for the two test set batches (batches 12 and 13): (**a**) Model 1, Batch 12; (**b**) Model 1, Batch 13; (**c**) Model 2, Batch 12; (**d**) Model 2, Batch 13; (**e**) Model 3, Batch 12; (**f**) Model 3, Batch 13; (**g**) Model 4, Batch 12; (**h**) Model 4, Batch 13.

### **4. Discussion**

The most accurate model (Model 1) uses features from both the 1st and 2nd reflections and the process temperature. This shows the potential of US sensors to predict the endpoint of fermentation and, as demonstrated in Figure 5a,b, accurately predict the alcohol concentration throughout the fermentation process. However, industrial implementation of this model would require the use of an invasive probe in order to obtain the 2nd reflection. In addition, an invasive temperature probe would be required to monitor the changing temperature of the fermentation media. Interestingly, the second most accurate model (Model 4) only used features from the 1st reflection and excluded the process temperature. The use of only the 1st reflection indicates that accurate results could be obtained using a non-invasive, no-transmission US sensor, similar to the techniques used in previous works by our group [23–26]. This is advantageous as it allows the alcohol volume to be accurately predicted by easily mounting a US sensor externally to an existing vessel. Therefore, it can be easily implemented into existing industrial settings at low effort and cost. The use of Model 4 would also remove the requirement for an invasive process temperature measurement. Furthermore, the performance metrics for Model 4 (R2 = 0.948, MAE = 0.283, MSE = 0.146) are similar to those of Model 1 (R2 = 0.952, MAE = 0.265, MSE = 0.136) indicating that no prediction accuracy would be lost through using a non-invasive and no-transmission sensor approach. This US sensing technique would also not require a hole to be bored into the vessel side, as used in this work. Instead, the US wave could be transmitted through the vessel wall.

In Model 3, the features from the 1st reflection combined with the process temperature produces a reduced accuracy. This is likely because the additional features required in Model 4 (the peak-to-peak amplitude, maximum amplitude, and minimum amplitude of the 1st reflection) contained more pertinent information about the temperature at the probewort interface than the non-local temperature sensor. The suggestion that the temperature sensor measured the temperature of the bulk wort instead of the region through which the 1st reflection passes is supported by the results from Model 2. When the temperature was removed as a feature, Model 2 produced a reduced accuracy compared with Model 1. This indicates that for accurate prediction using the 2nd reflection, the bulk wort temperature measurement is required as the sound wave travels through this region. The reduced accuracy obtained when combining the temperature data with the 1st reflection for Model 3 is most likely caused by the temperature at the probe-wort interface not closely following the trend of the bulk wort temperature. Therefore, using the temperature measurement as a feature increases the model complexity with little benefit, meaning it is more difficult for the network to find an optimal solution. This further supports the aforementioned point that accurate, invasive temperature measurement would not be required with a non-invasive, no-transmission US sensing technique.

Figure 5 displays the predicted ABV percentage from the trained LSTM models for the two batches used for the test set (batches 12 and 13). Model 1 (Figure 5a and b) accurately determines the fermentation endpoint. However, the final ABV prediction is not as accurate as Model 4, indicating that it may not be sensitive enough to determine differences in final ABV between batches. Whilst Model 3 appears to have no utility, Models 1, 2, and 4 all accurately followed the ABV trajectory. Owed to the real-time data acquisition of US sensors, these models suggest that the obtained data could be used to train additional anomaly detection models to provide early warning of undesired process trajectories within a batch.

Several locations in the prediction require improvement; for example, the detection of ABV plateau for Model 2 (Figure 5c,d around the 2nd day), the settling at a final ABV for Model 3 (Figure 5e,f), and the detection of the initial ABV rise for Model 4 (Figure 5g,h around the first day). This is likely due to the varying temperature throughout the fermentation having a large effect on the US properties of the wort compared with the changing density. There are also locations of decreasing ABV prediction (Figure 5d during the first day) or sudden increases in ABV prediction (Figure 5h at the end of the fourth

day). This is likely due to the temperature variations being different for each batch and the particular temperature variations during the test set causing these effects. These problems would likely be reduced through obtaining more training data.

In this work, ML models were trained to predict the ABV throughout the fermentation. However, in industrial settings this may not be the most appropriate output value with which to fit a model. For example, ML models could be trained to predict the final ABV of each batch, the time remaining until the ABV plateaus, classify the end of fermentation, or provide early detection of anomalous batches. In each of these cases, the models would be trained for a more specific purpose, as such the models may perform better than indicated by Figure 5. This work is therefore demonstrative of the efficacy of real-time fermentation monitoring using US sensors and ML, and increased accuracy may be achieved through predictions of more specific outputs.

If only the 1st reflection was used in an industrial monitoring system, the sound wave could be transferred through the vessel wall. Alternatively, if the 2nd reflection was also to be used, the probe could be fitted through existing ports common to industrial fermenters. This work monitored a laboratory scale fermentation process. At industrial scale, agitation methods are uncommon in beer fermentation to prevent damage to the yeast [36]. Therefore, radial variations in alcohol concentration exist and there would be a difference in the alcohol concentration at the sensor measurement area and the bulk wort [36]. However, previous work from our group showed that a non-local probe could accurately monitor a mixing process [23]. This is because, through machine learning, the sensor data is correlated to the location of the ground truth data, rather than the sensor. In this case, a sensor would be trained to predict the alcohol concentration at the location of the hydrometer measurements or sample collection.

### *Future Research Directions*

The largest barrier to industrial implementation of sensors and ML combined technologies is the burden of obtaining labelled data. Labelled data is used as the targets for training supervised ML models. To obtain the ground truth to label data requires another analysis method. In this work, the hydrometer readings were used due to the sample density measurements producing insufficient data points and disturbing the fermentation. In an industrial setting, the hydrometer may only be able to be used for a small number of batches for ML model development. In this case, semi-supervised learning may be required to train high accuracy models. Semi-supervised learning uses both labelled and unlabelled samples to train a model [37–39]. Firstly, unsupervised learning techniques, such as principal component analysis or autoencoders, can be used on the total dataset to learn relationships between features across the labelled and unlabelled samples. Then traditional supervised learning can be used on the new features using just the labelled samples. Secondly, a self-training (or pseudo-labelling) approach may be used to predict the labels of the unlabelled data from the trained model. These pseudo-labels may then be added to the labelled data set and the procedure repeated to improve the label predictions or to train a final model.

Alternatively, conventional sample extraction and density measurement may be used to obtain the labelled data. Either a curve may be fitted to these sparse density measurements to produce interpolated data points, or a similar semi-supervised learning procedure can be implemented. Active learning may also be used to identify data points for labelling that may be the most useful to the model [40,41]. These datapoints may be during a sparsely sampled time in the fermentation or be in a particular temperature and composition range. Operators could then analyse these samples to provide the most benefit to the ML model at the lowest investment in effort.

### **5. Conclusions**

The transition to Industry 4.0 promises increased manufacturing efficiency, sustainability, and productivity. By implementing digital technologies such as the Internet of

Things, Cloud Computing and ML, not only can entire processes be integrated, but supply chains as well. Sensors are a key technology in this revolution by providing the real-time data to inform automatic, intelligent decision-making. Currently, beer fermentation is monitored through periodic manual sampling and off-line wort density measurements. This work has presented an in-line, low-cost US sensing technique combined with ML, which would remove the need of operator sampling. This work has shown that US sensor data combined with LSTM models are able to accurately predict the volume of alcohol during beer fermentation. The highest accuracy model (R2 = 0.952) used a transmissionbased ultrasonic sensing technique along with the process temperature. Importantly, the second most accurate model (R2 = 0.948) only used a reflection-based technique without measurement of the temperature. This demonstrates the potential for a non-invasive, no-transmission US technique, which doesn't require invasive measurement of the process temperature. This sensing technique could be easily and inexpensively retrofitted onto existing fermentation vessels.

**Author Contributions:** Conceptualization, A.B., J.E. and N.W.; methodology, A.B., J.E., N.W. and M.P.; software, A.B. and J.E.; validation, A.B.; formal analysis, A.B.; investigation, A.B. and J.E.; resources, A.B.; data curation, A.B. and J.E.; writing—original draft preparation, A.B.; writing review and editing, A.B., N.W. and M.P.; visualization, A.B.; supervision, N.W.; project administration, N.W.; funding acquisition, N.W. All authors have read and agreed to the published version of the manuscript.

**Funding:** This work was supported by the Engineering and Physical Sciences Research Council (EPSRC) standard research studentship (EP/R513283/1) and EPSRC network+ Connected Everything (EP/P001246/1).

**Data Availability Statement:** Researchers at the University of Nottingham can be contacted for access to data.

**Conflicts of Interest:** The authors declare no conflict of interest.

### **References**

### *Article* **Determination of Foam Stability in Lager Beers Using Digital Image Analysis of Images Obtained Using RGB and 3D Cameras**

**Emmanuel Karlo Nyarko 1 , Hrvoje Glavaš 1,\*, Kristina Habschied 2,\* and Krešimir Mastanjevi´c 2**

**Abstract:** Foam stability and retention is an important indicator of beer quality and freshness. A full, white head of foam with nicely distributed small bubbles of CO2 is appealing to the consumers and the crown of the production process. However, raw materials, production process, packaging, transportation, and storage have a big impact on foam stability, which marks foam stability monitoring during all these stages, from production to consumer, as very important. Beer foam stability is expressed as a change of foam height over a certain period. This research aimed to monitor the foam stability of lager beers using image analysis methods on two different types of recordings: RGB and depth videos. Sixteen different commercially available lager beers were subjected to analysis. The automated image analysis method based only on the analysis of RGB video images proved to be inapplicable in real conditions due to problems such as reflection of light through glass, autofocus, and beer lacing/clinging, which make it impossible to accurately detect the actual height of the foam. A solution to this problem, representing a unique contribution, was found by introducing the use of a 3D camera in estimating foam stability. According to the results, automated analysis of depth images obtained from a 3D camera proved to be a suitable, objective, repeatable, reliable, and sufficiently sensitive method for measuring foam stability of lager beers. The applied model proved to be suitable for predicting changes in foam retention of lager beers.

**Keywords:** foam stability; image analysis; lager beer; foam retention

### **1. Introduction**

Ancient beer displayed weak or no foam. Uncontrolled fermentation, no addition of hops, and subsequent low carbonation resulted in a foam-less beverage. Research conducted by [1] on ancient Finnish beer Sahti showed that the investigated ancient beverage had no foam and showed a distinctive difference between today's beers, especially regarding flavor and aroma. Today's brewing industries are far from the ancient manufacturers, and stable and retentive foam head is one of the main indicators of beer freshness and quality. Even though a big and rich head of foam is a property of certain types of beer (lager, pilsner, and wheat beer among others), every consumer seeks freshness in a preferable label. Some people do not appreciate foam in their glass, regardless of the beer style, some love the lacy pattern at the bottom of the finished beer, but the majority of beer-lovers like the crystal-clear glass after finishing the last sip [2]. Cling can be described as the adhesion of beer foam to the side of the glass during beer consumption, commonly known as "lacing". For example, Belgium is known for beers that leave a lacy glass. According to BJCP Beer Style Guidelines [3], "Belgian Lace is a characteristic and persistent latticework pattern of foam left on the inside of the glass as a beer is consumed. The look is reminiscent of fine lacework from Brussels or Belgium, and is a desirable indicator of beer quality

65

**Citation:** Nyarko, E.K.; Glavaš, H.; Habschied, K.; Mastanjevi´c, K. Determination of Foam Stability in Lager Beers Using Digital Image Analysis of Images Obtained Using RGB and 3D Cameras. *Fermentation* **2021**, *7*, 46. https://doi.org/10.3390/ fermentation7020046

Academic Editor: Sigfredo Fuentes

Received: 21 February 2021 Accepted: 24 March 2021 Published: 26 March 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

in Belgium." According to Bamforth [2], foam quality is described by many properties such as stability, retention, viscosity, whiteness, bubble size, and density. As described by Gonzalez Viejo et al. [4], the most presentable indicators of foam behavior are foamability (capacity of foam formation) and foam stability. Many factors affect foam stability and the physics behind the foam is extensive, but in short, foams are colloids comprising gas bubbles dispersed in liquid [2]. A detailed description of the physics of foam formation and stability is well described by Bamforth [2] and Hackbarth [5]. Many other authors also did an excellent job in reporting and describing the physics of foam [6–9]. A more recent contribution was provided by Gonzalez Viejo et al. [4] in an extensive review. According to [5], a decrease in surface tension reduces foam stability. This can be influenced by high fermentation temperatures resulting in fusel oils accumulation, yeast autolysis (re-leases lipids and protease A), an unpasteurized beer that allows protease A, or unclean or improperly rinsed serving glasses. Agents that aid foam stability are high molecular weight (MW > 5000) malt proteins (Z4, LTP1, lipid binding indolines, and hordeins). The choice of raw material, malting process conditions, kilning temperatures, mashing-in temperatures, excessive boiling, and excessive use of chillproofing filter-aids can influence the foam stability. A reaction between malt proteins and isomerized hop alpha acids also helps in foam stability.

Due to its importance for the brewing industry, many scholars and professionals came up with several methods for the determination and measurement of foam stability and quality [10–21]. Many of these properties, such as determination of foam stability or head retention, lacing, bubble size, whiteness, foam density, foam viscosity, foam strength, etc., are well described in a review paper by Bamfort [2]. However, modern methodologies and approaches tend to be less or even non-invasive and are therefore suitable for application in foam measurements [22–25]. Among the most popular methods are currently image analysis methods. These methods, however, have some drawbacks, especially in scenarios with automated procedures. The first problem is the reflection of light on the surface and within the beer glass, as reported by Lukinac et al. [26]. Another problem that may arise is focusing; the measuring set-up must be made under controlled directional light conditions that entail lower illuminance values and automated focusing. The third and biggest problem is foam lacing or clinging, which makes it impossible to detect the actual level of foam.

Electrode-involving method is excellent for measuring foam height (its decrease over time, and therefore its stability), but it demands regular cleaning and maintenance and is more time-consuming. The use of a camera in foam assessment demands no cleaning and is a low-maintenance, cheap, and easy method that requires minimal input from the employees, applicable in every laboratory. The implementation of a simple, easy, affordable, and non-invasive method that could solve or, at least, derail all of the above-mentioned problems would greatly help the brewing industry in an objective assessment of beer foam stability.

The aim of this paper was to analyze the applicability of an automated non-invasive, objective, and cheap image analysis method under real conditions, to follow up and measure the foam stability of lager beers produced and available on the Croatian market.

### **2. Materials and Methods**

Sixteen samples of commercially available light lager beers packaged in brown or green glass bottles (0.5 L) were set to be analyzed using the methods described below. Ten were domestic beers and six others were foreign (Germany, Czech Republic, Denmark, and Holland). Beer samples were held at room temperature (20 ◦C) for two days, in order to reduce the influence of temperature fluctuations. Glasses (0.5 L; generic brand, model Lilith, h = 185 mm, Ø = 0.75 mm) were bought, washed, and degreased then rinsed in demineralized water and left to dry two days prior to the analysis. All glasses were identical and held at room temperature for two days prior to analysis. Beer was hand-poured into a degreased glass, according to the standard MEBAK (Middle European Brewing Analysis

Commission) procedure (method 2.23.1) [27], at an angle of 135◦ . The pourer took extra care to pour every sample evenly and uniformly into the glass (Figure 1). When the pouring was done, the sample was placed on a designated spot, and cameras were set to measure the foam stability. According to MEBAK, foam in lager beers should be stable for 3–5 min. a visual RGB camera (Canon PowerShot G16; Ota City, Tokyo, Japan) was used to take a video recording of the beer. Simultaneously, a 3D camera (Orbbec Astra S; Orbbec 3D Technology International, Inc., Troy, MI, USA) was also used to take depth measurements. Figure 1 shows the experimental setup used in data collection.

Two vision-based approaches to measuring foam stability were implemented: image analysis of RGB video and depth measurement using a 3D camera. Over a period of 5 min,

into a degreased glass, according to the standard MEBAK (Middle European Brewing Analysis Commission) procedure (method 2.23.1) [27], at an angle of 135°. The pourer took extra care to pour every sample evenly and uniformly into the glass (Figure 1). When the pouring was done, the sample was placed on a designated spot, and cameras were set to measure the foam stability. According to MEBAK, foam in lager beers should be stable

*Fermentation* **2021**, *7*, x FOR PEER REVIEW 3 of 13

**Figure 1.** Experimental setup. **Figure 1.** Experimental setup.

for 3–5 min.

*2.1. Image Analysis of Video*  A visual RGB camera (Canon G16) was used to take a video recording of each sample over a period of 5 min. The recorded video had a frame rate of 30 fps. Image analysis was performed on the recorded video. The image analysis procedure to determine the height of beer foam is explained in the following seven steps below: Step 1: For a given frame of the recorded video, define a region of interest (ROI) of Two vision-based approaches to measuring foam stability were implemented: image analysis of RGB video and depth measurement using a 3D camera. Over a period of 5 min, a visual RGB camera (Canon PowerShot G16; Ota City, Tokyo, Japan) was used to take a video recording of the beer. Simultaneously, a 3D camera (Orbbec Astra S; Orbbec 3D Technology International, Inc., Troy, MI, USA) was also used to take depth measurements. Figure 1 shows the experimental setup used in data collection.

#### known width (*w*) and height (*l*) in the image that contains beer and foam, as shown in *2.1. Image Analysis of Video*

Figure 2a. It is important that the ROI covers the whole height of the foam and part of the beer. All the following image processing steps are performed on this ROI. This ensures that all the image processing steps are directed or focused on segmenting the foam head from the rest of the image within the ROI. A visual RGB camera (Canon G16) was used to take a video recording of each sample over a period of 5 min. The recorded video had a frame rate of 30 fps. Image analysis was performed on the recorded video. The image analysis procedure to determine the height of beer foam is explained in the following seven steps below:

Step 2: Perform color segmentation by filtering (thresholding) the ROI in HSV color space by defining the lower and upper values of the color of the foam (Figure 2b). HSV color space separates color information (chroma) from intensity (luma). Since the value is separated, thresholding can theoretically be performed using only saturation and hue. More robust color thresholding over simpler parameters can be performed in HSV color space than in RGB color space. For the purposes of the results presented in this paper, the Step 1: For a given frame of the recorded video, define a region of interest (ROI) of known width (*w*) and height (*l*) in the image that contains beer and foam, as shown in Figure 2a. It is important that the ROI covers the whole height of the foam and part of the beer. All the following image processing steps are performed on this ROI. This ensures that all the image processing steps are directed or focused on segmenting the foam head from the rest of the image within the ROI.

lower HSV boundary of (0,0,230)was used, while the upper boundary was defined as (43,18,255) Step 3: Generate a binary image of the thresholded ROI in HSV color space (Figures 2c and 3a), i.e., all pixels that have values within the defined boundaries are marked as white pixels (values are set to 255), while the remaining pixels are marked as black (values are set to 0). Step 2: Perform color segmentation by filtering (thresholding) the ROI in HSV color space by defining the lower and upper values of the color of the foam (Figure 2b). HSV color space separates color information (chroma) from intensity (luma). Since the value is separated, thresholding can theoretically be performed using only saturation and hue. More robust color thresholding over simpler parameters can be performed in HSV color space than in RGB color space. For the purposes of the results presented in this paper, the lower HSV boundary of (0,0,230)was used, while the upper boundary was defined as (43,18,255).

Step 3: Generate a binary image of the thresholded ROI in HSV color space (Figures 2c and 3a), i.e., all pixels that have values within the defined boundaries are marked as white pixels (values are set to 255), while the remaining pixels are marked as black (values are set to 0).

Step 4: Perform morphological operations of erosion followed by dilation on the binary image (Figures 2d and 3b). These operations are needed in order to eliminate small white noises or white artifacts that appear in the binary image.

Step 5: Determine the largest contour from the list of all contours on the binary image (Figure 2e). A contour is a curve joining all the continuous points or connected components (along the boundary) having the same color or intensity. This step basically segments or marks the boundary of the foam/head. *Fermentation* **2021**, *7*, x FOR PEER REVIEW 4 of 13

> Step 6: Determine the area (*A*) of the region defined by the largest contour. Step 4: Perform morphological operations of erosion followed by dilation on the bi-

Step 7: The average height (*h*) of the beer foam in pixels can be determined using Equation (1): nary image (Figures 2d and 3b). These operations are needed in order to eliminate small white noises or white artifacts that appear in the binary image.

$$h = \mathbf{A}/\mathbf{w}.\tag{1}$$

Using the notation where *ht* represents the height of foam (in pixels) at a given point in time *t* (in seconds), the maximum height, *hmax*, is defined as: (Figure 2e). A contour is a curve joining all the continuous points or connected components (along the boundary) having the same color or intensity. This step basically seg-

$$h\_{\max} = \max \left\{ h\_l \colon t = 0, \ 1, \ldots, \ \Re 00 \right\}, \tag{2}$$

and the minimum height, *hmin*, is defined as Step 7: The average height (*h*) of the beer foam in pixels can be determined using Equation (1):

ments or marks the boundary of the foam/head.

in time *t* (in seconds), the maximum height, *hmax*, is defined as:

$$h\_{\min} = \min \left\{ h\_t \colon t = 0, 1, \ldots, \Re 00 \right\}. \tag{3}$$

Based on these definitions, we define the normalized foam height at time *t*, *ht\_norm*, as: Using the notation where *ht* represents the height of foam (in pixels) at a given point

$$h\_{\text{t\\_norm}} = (h\_{\text{t}} - h\_{\text{min}}) / (h\_{\text{max}} - h\_{\text{min}}).\tag{4}$$

Since the beer glass is always located in the same position, the seven steps provided above can be implemented as a program to automate the procedure. One advantage of this procedure is that it can be run in both offline and online mode. For the purposes of this paper, a script written in the Python programming language [28] using the OpenCV library [29] was implemented in order to automate the process of determining the beer foam height from the recorded videos. Every 10 s, five consecutive frames were taken and the height of foam determined for each frame. The average height in pixels for these five measurements was taken to represent the height of foam every 10 s. and the minimum height, *hmin*, is defined as *hmin* = min {*ht*: *t* = 0, 1, …,300}. (3) Based on these definitions, we define the normalized foam height at time *t*, *ht\_norm*, as: *ht\_norm* = (*ht* − *hmin*)/(*hmax* − *hmin*). (4)

**Figure 2.** Estimating foam height from an image. (**a**) Region of interest (ROI) of known width (*w*) and height (*l*) defined for the image. The ROI is marked in blue; (**b**) thresholding of ROI performed in HSV color space. The pixels satisfying the threshold are marked in red; (**c**) binary image of ROI thresholded in HSV color space (a magnified image is provided in Figure 3a); (**d**) morphological operations of erosion followed by dilation performed on binary image to remove artifacts (a magnified image is provided in Figure 3b); (**e**) largest contour found marked in red. The estimated height of foam in pixels is determined using Equation (1). **Figure 2.** Estimating foam height from an image. (**a**) Region of interest (ROI) of known width (*w*) and height (*l*) defined for the image. The ROI is marked in blue; (**b**) thresholding of ROI performed in HSV color space. The pixels satisfying the threshold are marked in red; (**c**) binary image of ROI thresholded in HSV color space (a magnified image is provided in Figure 3a); (**d**) morphological operations of erosion followed by dilation performed on binary image to remove artifacts (a magnified image is provided in Figure 3b); (**e**) largest contour found marked in red. The estimated height of foam in pixels is determined using Equation (1).

**Figure 3.** Magnified images of Figure 2a,b: (**a**) Initial binary image of ROI thresholded in HSV color space with visible artifacts; (**b**) Final binary image of ROI thresholded in HSV color space with artifacts removed after performing morphological operations. **Figure 3.** Magnified images of Figure 2a,b: (**a**) Initial binary image of ROI thresholded in HSV color space with visible artifacts; (**b**) Final binary image of ROI thresholded in HSV color space with artifacts removed after performing morphological operations. given by *difft* = *dtable* − *dt*. (5)

The maximum height of the foam from the top of the table, *diffmax*, is defined as:

#### above can be implemented as a program to automate the procedure. One advantage of *2.2. Depth Measurement Using a 3D Camera*

this procedure is that it can be run in both offline and online mode. For the purposes of this paper, a script written in the Python programming language [28] using the OpenCV library [29] was implemented in order to automate the process of determining the beer foam height from the recorded videos. Every 10 s, five consecutive frames were taken and the height of foam determined for each frame. The average height in pixels for these five measurements was taken to represent the height of foam every 10 s. *2.2. Depth Measurement Using a 3D Camera* A 3D camera provides a depth map or a depth image where each pixel in the image A 3D camera provides a depth map or a depth image where each pixel in the image relates to the distance between the surface of the object being viewed and the camera or image plane. The Orbbec Astra S 3D camera used in this paper is based on the Structured-Light technology. The 3D camera consists of an infrared laser projector and a proprietary Infra-Red (IR) depth sensor. The depth sensor interprets 3D scene information based on continuously projected infrared structured light. It should also be mentioned that Orbbec Astra S 3D camera also consists of an RGB camera. However, this RGB camera was not used in the experiments for the purpose of this paper. An example of a depth image generated by the 3D camera is shown in Figure 4. *diffmax* = max {*difft*: t = 0, 1, …,300}, (6) and the minimum height, *diffmin*, is defined as *diffmin* = min {*difft*: t = 0, 1, …,300}. (7) Based on these definitions, the normalized foam height at time *t*, *ht\_norm*, is given by: *ht\_norm* = (*difft* − *diffmin*)/(*diffmax* − *diffmin*). (8)

relates to the distance between the surface of the object being viewed and the camera or

Since the beer glass is always located in the same position, the seven steps provided

the foam head from the 3D camera. Thus, measurements of the distance of the foam for a given sample increased with time, as the foam in the glass decreased. Similar to the measurements performed when using the video camera in the previous section, since the beer glass was always located in the same position, a fixed ROI was **Figure 4.** Examples of depth images obtained using Orbbec Astra S 3D camera. Black pixels indicate that no depth information exists: (**a**) ROI defined in the middle of the glass containing foam; (**b**) the same view after a few minutes when the foam disappears. **Figure 4.** Examples of depth images obtained using Orbbec Astra S 3D camera. Black pixels indicate that no depth information exists: (**a**) ROI defined in the middle of the glass containing foam; (**b**) the same view after a few minutes when the foam disappears.

defined (Figure 4a), and a Python script was used in order to automate the process of determining the distance of the beer foam height from the camera. Every 10 s, five consecutive frames were taken, and the average distance of foam from the camera was determined for each frame. This average distance was determined using only the pixels within It is important to emphasize that the images displayed in Figure 4 represent depth images. Areas in the images having shades of gray have depth information, while those Orbbec Astra S depth sensor has a camera resolution of 640 × 480 and a maximum frame rate of 30 Hz. Its measurement range is from 0.4 to 2 m and has a field of view of 60◦ horizontally and 49.5◦ vertically. It also has an accuracy of +/−1–3 mm at 1 m.

the defined ROI having a depth value. Pixels without depth values were excluded. The

peared before 5 min.

**3. Results and Discussion**

since the emitter and the sensor on the 3D camera are separated by about 7.5 cm. In Figure 4b, after the foam disappears, the transmitted beam of the 3D sensor cannot be reflected by the beer surface, and therefore it is impossible to detect the depth of the beer surface. This criterion was used for ending measurements in situations when the foam disap-

RGB and depth video recordings were obtained for 16 samples (denoted by s01…s16), each lasting 5 min. Using the procedures described in Sections 2.1 and 2.2, the estimated height (in pixels) and distance of foam from the camera (in mm) were obtained from the RGB video and depth video, respectively, every 10 s. The results of the measurements are displayed in Tables 1 and 2. Figures 5 and 6 display the corresponding normalized foam heights (%) obtained by performing image analysis on RGB images and from

marked in black do not. The edge of the beer glass cannot be detected in Figure 4a due to

One drawback of the sensor is that it cannot detect glass nor liquids, so for example, in Figure 4, it can be seen that the edge of the beer glass cannot be detected (pixels displayed in black), and when the foam disappears no depth information can be obtained.

As displayed in Figure 1, the 3D camera was placed above the beer glass. Hence, all measurements obtained from the 3D camera actually provided the distance of the top of the foam head from the 3D camera. Thus, measurements of the distance of the foam for a given sample increased with time, as the foam in the glass decreased.

Similar to the measurements performed when using the video camera in the previous section, since the beer glass was always located in the same position, a fixed ROI was defined (Figure 4a), and a Python script was used in order to automate the process of determining the distance of the beer foam height from the camera. Every 10 s, five consecutive frames were taken, and the average distance of foam from the camera was determined for each frame. This average distance was determined using only the pixels within the defined ROI having a depth value. Pixels without depth values were excluded. The mean distance of these average distances for the five measurements was taken to represent the distance of foam from the camera every 10 s.

If *dt* represents the distance of foam (in mm) from the camera at a given point in time (t = 0, 1, . . . ,300 s), and *dtable* represents the distance of the table (in mm) from the camera (*dtable =* 654 mm. Figure 1.), the height of the foam from the top of the table at time *t*, *difft* , is given by

$$\text{diff}\_{\text{t}} = d\_{\text{table}} - d\_{\text{t}}.\tag{5}$$

The maximum height of the foam from the top of the table, *diffmax*, is defined as:

$$diff\_{\max} = \max \left\{ diff\_{\text{f}} \colon \mathbf{t} = \mathbf{0}, \mathbf{1}, \dots \text{ } \text{\textquotedblleft} \mathbf{0} \mathbf{0} \right\} \tag{6}$$

and the minimum height, *diffmin*, is defined as

$$diff\_{\min} = \min \left\{ diff\_l \colon \mathbf{t} = \mathbf{0}, \ 1, \ \dots \ \mathbf{\mathcal{Bool}} \right\}. \tag{7}$$

Based on these definitions, the normalized foam height at time *t*, *ht\_norm*, is given by:

$$h\_{\text{\\_norm}} = (\text{diff}\_{\text{\\_}} - \text{diff}\_{\text{\\_min}}) / (\text{diff}\_{\text{max}} - \text{diff}\_{\text{\\_min}}).\tag{8}$$

It is important to emphasize that the images displayed in Figure 4 represent depth images. Areas in the images having shades of gray have depth information, while those marked in black do not. The edge of the beer glass cannot be detected in Figure 4a due to the fact that (a) the sensor cannot detect glass, since the transmitted light is not reflected, and (b) the non-defined area near the beer glass is also extended as a result of parallax, since the emitter and the sensor on the 3D camera are separated by about 7.5 cm. In Figure 4b, after the foam disappears, the transmitted beam of the 3D sensor cannot be reflected by the beer surface, and therefore it is impossible to detect the depth of the beer surface. This criterion was used for ending measurements in situations when the foam disappeared before 5 min.

### **3. Results and Discussion**

RGB and depth video recordings were obtained for 16 samples (denoted by s01 . . . s16), each lasting 5 min. Using the procedures described in Sections 2.1 and 2.2, the estimated height (in pixels) and distance of foam from the camera (in mm) were obtained from the RGB video and depth video, respectively, every 10 s. The results of the measurements are displayed in Tables 1 and 2. Figures 5 and 6 display the corresponding normalized foam heights (%) obtained by performing image analysis on RGB images and from depth images, respectively. The actual normalized values obtained are provided in Tables A1 and A2.

**Table 1.** Height of foam (pixels) determined by performing image analysis on RGB videos of 16 beer samples (the actual value in mm can be obtained using the conversion 1 mm = 6.6 px). bubble (disproportionation), so the more gas there is in foam, the greater the dispropor-

*Fermentation* **2021**, *7*, x FOR PEER REVIEW 8 of 13

"-" indicates that no measurements were made since there was no foam head. All values have been rounded up. normal behavior) lead us to this conclusion.

**Figure 5.** Normalized foam height (%) determined by performing image analysis on RGB videos of 16 beer samples. **Figure 5.** Normalized foam height (%) determined by performing image analysis on RGB videos of 16 beer samples.

**Table 2.** The distance of foam head surface (mm) from the top of the table for 16 beer samples.

"-" indicates that no measurements were made since there was no foam head. All values have been rounded up.

**Figure 6.** Changes of foam head of sample s02 over time due to the erratic behavior of CO2 bubbles. **Figure 6.** Changes of foam head of sample s02 over time due to the erratic behavior of CO2 bubbles.

Depth measurements were also being taken simultaneously using a 3D camera. The distance of the beer foam surface (in mm) from the top of the table measured over time, *difft*, for the 16 beer samples is displayed in Table 2. The sign "-" indicates that measurements were not possible since there was no foam head. The processed results of the measurements obtained using the visual RGB camera are displayed in Table 1. Even though the results are displayed in pixels, the corresponding height of beer foam can be obtained by using the conversion of 1 mm = 6.6 px. The sign "-" in the table indicates that measurements were stopped since there was no foam.

**Table 2.** The distance of foam head surface (mm) from the top of the table for 16 beer samples. **Time (s) s01 s02 s03 s04 s05 s06 s07 s08 s09 s10 s11 s12 s13 s14 s15 s16** 0 158 159 161 172 171 177 181 176 178 167 177 170 134 168 165 191 A graphical representation of the normalized measurement results obtained by performing image analysis on RGB videos can be seen in Figure 5 (actual data is provided in Table A1). The behavior of the s02 sample is due to the increase in foam levels during the measurement as a result of erratic CO2 bubbles that formed unstable foam, as can be

"-" indicates that no measurements were made since there was no foam head.  According to Bamforth [2], low surface tension is an important factor for foam formation. Constant surface tension withholds a pressure within a bubble that is inversely proportional to its diameter, so the gas makes an effort to pass from a smaller to larger bubble (disproportionation), so the more gas there is in foam, the greater the disproportionation, which was the case for most samples, but sample s02 was particularly erratic. If the gas fraction in liquid (beer) foams is high, the bubbles cannot form exclusively spheres, but they take forms of polyhedra separated by thin layers of the liquid phase called lamellae. Another important phenomenon is that hydrophobic particles adsorbed at the gas–liquid interface tend to compress together as bubbles contract to form barriers that prevent the continuation of disproportionation. At constant pressure, the size of bubbles is directly proportional to the surface tension. Thus, materials with lower surface tension also give smaller bubbles [2]. Coalescence or merging of two bubbles occurs upon rupture of the membrane that divides them. This leads to coarsening of foam with visible larger bubbles–fish eyes in the foam body [4]. The Young–Laplace equation describes the disproportionation as the differential pressures between the inside and outside of a bubble due to surface tension. This pressure is inversely proportional to the bubble radius, causing CO2 gas to diffuse from smaller bubbles where pressure is higher into larger bubbles. According to Hackbath [4], "as the foam structure coarsens and larger bubbles continue to expand, their membranes thin until they reach a critical thickness. Film ruptures can be spontaneous or can be caused by fats that interfere with the film's external surface. Collapse occurs at the crown surface by rupture or by diffusion of dipolar CO2 directly to the atmosphere through the CO2 permeable bubble film". Comprehension of all stated data could explain the behavior of foam in sample s02. It can be presumed that this is due to the storage in unsuitable conditions in the supermarket storage space. All samples were purchased in January, when it was cold in the storage rooms of the market place, and all analyses were done in January. The temperature fluctuations in the storage room, where it is cooler, then sudden transfer to higher temperatures at the market place could cause this kind of foaming properties loss in most of the samples. As for the sample s02, it could be some type of production error in this particular batch. Apart from sample s02, samples s16 and s07 seemed to show a more stable foam in comparison to all the other samples. It appears that this foam showed significantly more stable properties, even though all samples were kept at the same temperature. This hypothesis has yet to be confirmed by detailed laboratory testing, although preliminary analysis of new samples obtained in March (which show normal behavior) lead us to this conclusion.

Depth measurements were also being taken simultaneously using a 3D camera. The distance of the beer foam surface (in mm) from the top of the table measured over time, *difft* , for the 16 beer samples is displayed in Table 2. The sign "-" indicates that measurements were not possible since there was no foam head.

A graphical representation of the normalized distance of beer foam (%) from 3D camera can be seen in Figure 7 (actual data is provided in Table A2). Comparing Figures 5 and 6, similar conclusions about the foam stability can be made.

One major drawback of the non-invasive automated image analysis of RGB images is that it is sensitive to foam lacing or clinging. For example, s06 has a foam height of 26 px (or about 4 mm) after 40 s (see Table 1). Figure 8 shows the RGB video frame after 40 s. On the other hand, depth measurements of the foam surface by the 3D camera were not possible after 30 s, since there was no foam on the liquid surface (scenario similar to Figure 4b). This feedback (lack of depth information) from the 3D sensor was then used to stop further measurement. It should also be noted that the measurements after 10 s (see Table 2) for samples s01 and s13 should basically be ignored, since this were unreliable measurements provided by the 3D sensor in situations where there was basically no foam.

A graphical representation of the normalized distance of beer foam (%) from 3D camera can be seen in Figure 7 (actual data is provided in Table A2). Comparing Figures 5 and

A graphical representation of the normalized distance of beer foam (%) from 3D camera can be seen in Figure 7 (actual data is provided in Table A2). Comparing Figures 5 and

6, similar conclusions about the foam stability can be made.

*Fermentation* **2021**, *7*, x FOR PEER REVIEW 10 of 13

6, similar conclusions about the foam stability can be made.

**Figure 7.** A normalized distance of beer foam (%) from 3D camera for 16 beer samples. **Figure 7.** A normalized distance of beer foam (%) from 3D camera for 16 beer samples. 2) for samples s01 and s13 should basically be ignored, since this were unreliable meas-

urements provided by the 3D sensor in situations where there was basically no foam.

One major drawback of the non-invasive automated image analysis of RGB images

**Figure 8.** Sample s06 after 40 s. (**a**) Foam is detected by automated image analysis of RGB image, even though this is just foam clinging to the beer glass; (**b**) 3D camera records this as lack of foam. **Figure 8.** Sample s06 after 40 s. (**a**) Foam is detected by automated image analysis of RGB image, even though this is just foam clinging to the beer glass; (**b**) 3D camera records this as lack of foam.

### **4. Conclusions**

(**a**) (**b**) **Figure 8.** Sample s06 after 40 s. (**a**) Foam is detected by automated image analysis of RGB image, even though this is just foam clinging to the beer glass; (**b**) 3D camera records this as lack of foam. Beer foam stability is an important beer quality indicator. Stable beer foam after production does not have to correlate with beer foam after a certain period of storage and transport. In this research, we presented an algorithm for an automated non-invasive procedure for measuring foam height by applying image analysis of RGB images or videos. The procedure showed off as relatively robust and applicable in online and offline mode. One major drawback of this method appeared to be its sensitivity to foam lacing or cling due to poor CO2 distribution or low foam active/stabilizing compounds concentrations (proteins or hop compounds) in beer where the camera, placed laterally in regards to the sample, could not distinguish the lacing from foam height. However, this problem was resolved by using a 3D camera, which generates depth videos or images. A 3D camera mounted directly above the glass containing the beer sample was able to measure the distance of the foam surface from the camera. By measuring the change in the distance of the foam surface from the 3D camera over time, information about the foam stability was

obtained. Due to the technology used in measuring the distance of objects from the 3D camera, the camera was able to detect and recognize the foam surface, but was not able to detect the liquid surface, and therefore the distance of the liquid surface from the camera could not be measured. This usual drawback of this camera is actually an advantage in this scenario, since it provides information about the disappearance of the beer foam. Information about the lack of foam is triggered by the lack of depth information within the ROI. In any case, this could be a novel, quick, robust, precise, and accurate method for foam stability measurement.

**Author Contributions:** Conceptualization: K.H. and E.K.N.; methodology: E.K.N. and H.G.; software: E.K.N.; formal analysis: H.G.; investigation: K.M.; writing—original draft preparation: K.H.; writing—review and editing: E.K.N. and K.M. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** The data presented in this study are available on request from the corresponding author.

**Conflicts of Interest:** The authors declare no conflict of interest.

### **Appendix A**

**Table A1.** Normalized foam height (%) determined by performing image analysis on RGB videos of 16 beer samples.

**Table A1.** *Cont.*

"-" indicates that no measurements were made since there was no foam head. All values have been rounded up.

**Table A2.** Normalized distance of beer foam (%) from 3D camera for 16 beer samples.

"-" indicates that no measurements were made since there was no foam head. All values have been rounded up.

### **References**

### *Article* **Yeast Morphology Assessment through Automated Image Analysis during Fermentation**

**Mario Guadalupe-Daqui, Mandi Chen, Katherine A. Thompson-Witrick and Andrew J. MacIntosh \***

Food Science and Human Nutrition Department, University of Florida, Gainesville, FL 32611, USA<EMAIL_ADDRESS>(M.G.-D<EMAIL_ADDRESS>(M.C<EMAIL_ADDRESS>(K.A.T.-W.) **\*** Correspondence<EMAIL_ADDRESS>Tel.: +1-352-294-3594

**Abstract:** The kinetics and success of an industrial fermentation are dependent upon the health of the microorganism(s) responsible. *Saccharomyces* sp. are the most commonly used organisms in food and beverage production; consequently, many metrics of yeast health and stress have been previously correlated with morphological changes to fermentations kinetics. Many researchers and industries use machine vision to count yeast and assess health through dyes and image analysis. This study assessed known physical differences through automated image analysis taken throughout ongoing high stress fermentations at various temperatures (30 ◦C and 35 ◦C). Measured parameters included sugar consumption rate, number of yeast cells in suspension, yeast cross-sectional area, and vacuole cross-sectional area. The cell morphological properties were analyzed automatically using ImageJ software and validated using manual assessment. It was found that there were significant changes in cell area and ratio of vacuole to cell area over the fermentation. These changes were temperature dependent. The changes in morphology have implications for rates of cellular reactions and efficiency within industrial fermentation processes. The use of automated image analysis to quantify these parameters is possible using currently available systems and will provide additional tools to enhance our understanding of the fermentation process.

**Keywords:** yeast morphology; automated image analysis; heat stress; vacuoles; cell size; computer vision

### **1. Introduction**

The fermentative properties of yeast have been utilized for thousands of years in a spectrum of applications including brewing, baking, biofuels, etc. For an optimal performance in most applications, the health of the yeast should be maintained throughout the fermentation [1]. Yeast health is typically measured through the determination of viability and vitality via metabolic dyes, however, there are other known morphological characteristics that have been shown to be indicators of yeast health [2,3] including size, number and shape of organelles (i.e., vacuoles). To optimize the fermentation process and quickly respond to deviations, data collection concerning yeast condition should be rapid and include automation as to be completed in a timely manner. If implemented, automated assessment can also provide supplemental information including details of fermentation rate per suspended cell which may assist both researchers and industry to better understand the fermentative process. This study used an open-source image processing to track changes in yeast morphology through an automated analysis of various characteristics during high-stress fermentations.

Commonly measured morphological characteristics of yeast include cell size (commonly expressed as cross-sectional area or volume) and the number of buds. These characteristics are used to evaluate yeast cells under a wide variety of environmental stress sources, for example, yeast cell size measurements have been used by various researchers [2,4] as an indicator of the effects of stressors such as temperature and ethanol. Additionally, cell size and the number of budding vs. single cells were successfully used to

**Citation:** Guadalupe-Daqui, M.; Chen, M.; Thompson-Witrick, K.A.; MacIntosh, A.J. Yeast Morphology Assessment through Automated Image Analysis during Fermentation. *Fermentation* **2021**, *7*, 44. https:// doi.org/10.3390/fermentation 7020044

Academic Editor: Claudia Gonzalez Viejo

Received: 5 March 2021 Accepted: 20 March 2021 Published: 24 March 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

assess the effects of hyperbaric stress and various gas compositions on yeast physiological state [5]. The vacuolar structure (size and shape) and number are other yeast morphological characteristics commonly assessed. Yeast vacuoles are dynamic organelles with a changing morphology during fermentation as a response to different stress sources [6–8]. These researchers studied yeast vacuole formation under osmotic and ethanol stress environments during fermentation at low temperatures. Both studies concluded that under high osmotic environments, each cell contained at least two or more yeast vacuoles which presented a fragmented (small diameter) structure. These studies also reported that yeast vacuoles formed under high ethanol concentrations were less fragmented and were described as single swollen vacuoles that occupied almost the entire cytoplasm. Furthermore, deficiencies of this organelle in yeast (observable under the microscope) have been linked to cell death [9] and the cessation of fermentation. A better understanding of yeast performance during fermentation can be attained by combining measurements of fermentation kinetics with morphological characteristics of the cell. Tibayrenc et al. [2] studied the evolution of *Saccharomyces cerevisiae* morphology under different stressors to infer cell viability. In summary, there have been numerous studies that effectively relate yeast morphological characteristics such as cell size, length, or state, to yeast properties during fermentation [2,10,11].

The automated analysis of images facilitates the examination of microorganisms at specific times throughout a process, providing instantaneous information concerning the parameters assessed. Typically, automated image analysis is divided into three stages: image processing, variable acquisition and statistical analysis [12]. Automated analysis of yeast cell images is not a novel concept and is most commonly used to assess the number of cells suspended within media. There are existing software and devices such as the Cellometer developed by Nexcelom Bioscience LLC (Lawrence, MA, USA) that can process yeast images and assess the number of cells and viability. CalMorph v1.3 is software created to analyze yeast cells delivering information regarding cell morphology, phase, etc. However, this software is limited to microscopic images that have been fluorescence stained [13]. There are numerous examples of researchers independently using various software programs to assess images of yeast for specific properties. For example, the software Matlab v.6.1 was used by Coelho et al. [5] to fully automate the image processing procedure from an image obtained using an optical microscope. These researchers studied the effects of hyperbaric stress on single and budding yeast cells using cell area size, major and minor axis length measured through Matlab. The open-source software ImageJ has been used as an image processing software by numerous researchers [14,15] to assess properties of various microorganisms. Stolze et al. [16] developed an automated counting method for Petrifilm plates using ImageJ, reducing the time of analysis significantly.

This study aimed to combine the use of automated image analysis with a modern understanding of how yeast morphology changes in response to adverse environmental conditions. This helped to correlate how changes in the morphological state of yeast influenced the process attributes throughout the fermentation. Specifically, fermentations were conducted at extreme temperature conditions (30 ◦C and 35 ◦C) where morphological changes were expected based upon previous work. Throughout the fermentation, hundreds of images were collected and attributes such as cell cross-sectional area and vacuole size were quantified through automated analysis. These were then correlated with fermentation kinetics to enhance understanding of the changes undergone by the yeast, and the effect upon the entire fermentation. The techniques used in this paper can easily be applied in the fermentation industries where automated data collection of this nature is often already taking place allowing rapid assessment of yeast adaptions to different stress sources.

### **2. Materials and Methods**

### *2.1. Media and Yeast*

The media used in this research was based upon a rum fermentation and was composed of 75% *w/w* distilled water, 18.5% *w/w* sugar cane, and 6.5% *w/w* blackstrap molasses. This broth was supplemented with 0.8%*w/w* yeast extract to increase the amount of free amino nitrogen to 320 mg/L (measured via Ninhydrin method as described in the American Society of Brewing Chemists (ASBC) Wort-12 method [17]). All the ingredients were dissolved at 50 ◦C and sterilized at 121 ◦C for 15 min. This media had a sugar concentration of 24% *w/w* and was used for both the propagation and fermentation processes. However, in the first propagation, the media was diluted to 15% *w/w* using deionized water. A *Saccharomyces cerevisiae* strain provided by Lallemand (DISTILAMAX® SR—Montreal, QC, Canada) was utilized throughout the experiments based upon recommendations from the supplier concerning survivability under adverse conditions and the ability to ferment sugars found in the raw materials used. The active dry yeast (ADY) was stored between 1 ◦C and 4 ◦C prior to its rehydration and propagation.

### *2.2. Rehydration and Propagation*

For each experiment, 10 g of ADY were rehydrated using 100 mL of sterile tap water at 35 ◦C for 15 min. Rehydrated yeast was propagated in a two-step process to achieve a consistent inoculum and sufficient biomass to pitch the 4 L fermentors. The first propagation was performed using 150 mL of media (diluted to 15% *w/w*), while the second propagation was performed using 350 mL of media at (24% *w/w*). Both propagation steps were pitched at 30 × 106 viable yeast cells/mL and run for 24 h at 35 ◦C each as per recommendations from the supplier.

### *2.3. Fermentation*

Each fermentation run started under identical conditions, using the same media, and pitched with the same yeast to assess the differences throughout the process by changing only the temperature. Fermentors (total volume of ~5 L) were filled with 4 L of media (24% *w/w*), oxygenated for two minutes, and pitched with 30 × 106 viable yeast cells/mL. Two temperatures were selected to complete the fermentation processes, 30 ◦C (low temperature) and 35 ◦C (high temperature). Each experiment was performed by duplicate. Samples were taken at intervals between 6 to 12 h in an attempt to monitor the reductions in the sugar concentration every 3–4% *w/w*, approximately, during peak fermentation. The volume of each sample was 125 mL and was divided into smaller aliquots to assess the fermentation attributes and the yeast morphology.

### *2.4. Measurement of Sugar Concentration and Viable Cells in Suspension*

To measure the sugar concentration throughout the fermentation, 30 mL of sample were degassed, centrifuged and filtered. These samples were analyzed using an Anton Paar Alex 500 (Graz, Austria), which measured the density of the sample through an oscillating U-tube glass. The density measured was then used to calculate the real sugar concentration and expressed in % *w/w*, or ◦P (analogous to percent sugar by weight). Cells in suspension were measured during the fermentation process using a hemocytometer and a microscope with 400× magnification, as described in the ASBC Yeast-4 method [18]. Viable cells were identified and counted using the methylene blue technique described by Painting and Kirsop [19]. The number of viable cells in suspension at each sample point was calculated by multiplying the percent of viable cells by the total number of cells in suspension.

### *2.5. Models*

The sugar concentration was modeled using the logistic model described in Yeast-14 [20] method and shown in Equation (1).

$$P\_{(t)} = P\_\varepsilon + \frac{P\_i - P\_\varepsilon}{1 + \exp^{(-B(t-M))}} \tag{1}$$

where P(t) is a function that represents the sugar concentration at a specific time t (% *w/w*), Pi is the upper asymptote of the curve (%*w/w*), Pe is the lower asymptote of the curve (%*w/w*), M is the time of the inflection point of the curve (h), and B is the slope of the curve

(%w/(w\*h)). To determine the consumption rate of the sugar during the fermentation, the first derivative of the logistic model (Equation (1)) was taken with respect to time (Equation (2)) as follows:

$$\frac{\partial}{\partial t} \left( P\_{\varepsilon} + \frac{P\_i - P\_{\varepsilon}}{1 + \exp^{\left(-B \ast (t - M)\right)}} \right) = \left( \frac{B \left( P\_i - P\_{\varepsilon} \right) \exp^{-B \left(x - M\right)}}{\left( \exp^{-B \left(x - M\right)} + 1 \right)^2} \right) \tag{2}$$

The cells in suspension data were modeled using the step model (Equation (3)) as described by Rudolph et al. [21]. This model was utilized due the amplitude and the Heaviside step function that describe the maximum number of cells in suspension during the fermentation and the final number of cells at the end of the process, respectively.

$$Y\_{(t)} = A \left( \frac{\exp^{-\frac{1}{2} \left( \frac{t-\mu}{1+(H\_t \ast S)} \right)^2}}{1+(H\_t \ast S)} \right) + \left( A - \frac{A}{(1+S)} \right) \ast H\_{(t)} \tag{3}$$

Y(t) is a function that represents the number of cells in suspension at a specific time t (cells/mL), A is the amplitude (cells/mL), m is the midpoint (h), σ is the width factor (h), S modifies the height of the step in this function (unitless), and H(t) is the Heaviside step function.

### *2.6. Image Collection*

Prior to the image collection, yeast cells were centrifuged and washed three times using deionized water and stained with a 0.01% *w/v* methylene blue solution to differentiate living and death cells at each data point. Various images were taken from each experiment such that approximately 80 to 360 cells were captured at each sampling point. Images were captured using a Nikon Eclipse Ci-L microscope at 1000× magnification, combining a 100× oil immersion objective and a 10× eyepiece.

### *2.7. Manual Analysis of Vacuole Count*

The manual analysis of the cell images was completed using the Nikon NIS-Elements software V5.30.01 compatible with the microscope. The cross-sectional area (CSA) of yeast and vacuoles were measured individually using an auto selection tool in conjunction with a five-point ellipse and a freehand selection tool. Once the areas were selected, the software converted the pixels into a measurement of surface area based upon previous calibrations. Budding yeast cells were assessed as two separate cells, dividing them at the cleavage. Vacuoles were clearly identifiable due to the difference in shade compared to the rest of the cell (Figure 1). The light settings on the microscope were adjusted at the beginning of the experiment to maximize this contrast. The data collected were exported and saved as an excel file for further calculations to determine the average yeast CSA and the ratio between vacuole and yeast CSA. Figure 1b shows an image analyzed manually and includes all the measurements taken using this software. There were 80–360 yeast cells assessed from images at the beginning, middle, and end of each fermentation process using this method to validate the automated image analysis method. Yeast cells that were partially in the image were not selected for data collection using this method.

**Figure 1.** (**a**) Microscope image of yeast during fermentation (1000×) with enlarged budding cell (counted as two individual yeast cells) to show shading difference between vacuole and yeast cell in a raw image; (**b**) The same image after manual analysis of cell and vacuole cross-sectional area (CSA).

### *2.8. Automated Image Analysis of Yeast and Vacuole Cross-Sectional Area*

The images of yeast taken throughout each fermentation were analyzed using ImageJ v1.8.0 software. A sample image is shown in Figure 2a. This analysis included the measurement of yeast cell CSA, total yeast CSA and total vacuole CSA. The automated image analysis was calibrated using a scale that correlated the image pixels with the scientific units of micrometers. The outline of yeast cells and the vacuoles were enhanced using tools sequentially: "binary mask", "despeckle filter", "dilation", and "pixel outlier removal". The goals of these tools were to reduce image noise that could affect the results and to enhance the distinguishing features of yeast cells from the background and the vacuoles. The binary mask converted the 32-bit image to 8-bit, which results in a black and white separation of the yeast cells (black), vacuoles (white) and background (white). The despeckle filter applied was a median filter that replaced every pixel with the median value of a 3 × 3 pixel region, and it removed sparsely occurring black pixels [22]. Then, the dilation tool was used to separate any yeast cells that may be touching each other due to proximity or budding. Finally, after dilation, the pixel outlier tool was applied to remove black particles with a radius of 4 pixels (0.363 µm) or less. A sample of the resulting processed images is shown in Figure 2b. These images were then automatically assessed with an analyze particles tool that selected for analysis, objects with an area of 10–100 µm2 and a circularity of 0.30–1.00 (ideally, this would encompass all yeast cells). This tool enumerated and collected the cross-sectional area of all individual selected objects (Figure 2c). To measure the vacuolar cross-sectional area, everything outside of the selected objects was removed, then the image was inverted to change the black background to white and white vacuoles to black, resulting in Figure 2d. Yeast cells and the vacuoles that were partially in the image were not included in the analysis by the software. The data collected were exported and saved as an excel file for subsequent calculation of the average yeast cell cross-sectional area and the ratio between vacuole and cell cross-sectional area. The automated analysis of the cell images was completed after the fermentation processes.

**Figure 2.** (**a**) Microscopic image of yeast cells (1000× magnification) at 6.5 h before processing; (**b**) After image processing; (**c**) After yeast cell area data were collected and region of interest established; (**d**) After clearing outside region of interest and inverting image and vacuolar area data was collected.

### *2.9. Statistical Analysis*

In this study, Minitab 19 was used to determine significant differences (*p* < 0.05) among the yeast CSA and CSA ratio between vacuoles and yeast cells throughout each fermentation and between fermentations at 30 ◦C and 35 ◦C. The validation of the automated image analysis method was performed by calculating the percentage error between the results obtained manually using the Nikon software V5.30.01 and automatically using ImageJ.

### **3. Results**

### *3.1. Fermentation Kinetics*

The sugar consumption model (Equation (1)) and the model for number of cells in suspension (Equation (3)) were fit to the data from each fermentation. The best fit model parameters for the combined fermentation data at each temperature are presented in Table 1. The sugar concentration regression resulted in high r2 values of 0.99, while the

slightly more variable cell in suspension data resulted in values of 0.91 and 0.85 for 30 and 35 ◦C, respectively.

Fermentations at 35 ◦C and 30 ◦C both showed a typical sigmoidal sugar consumption curve (Figure 3). The times of maximum fermentation rate (parameter M) were similar in both, at 30 ◦C (11.08 h) and 35 ◦C (11.39 h). At 30 ◦C, yeast metabolized more total sugars (20.8% *w/w*) compared to 35 ◦C (13.9% *w/w*). During fermentations at both temperatures, the number of viable cells in suspension increased (Figure 4). However, the maximum number of cells in suspension during the 35 ◦C fermentations was approximately 50% fewer compared to the 30 ◦C fermentations. These results are in accordance with the results previously reported by Torija et al. [23] and Reddy and Reddy [24] who used *Saccharomyces cerevisiae* strains to study the influence of temperature on the number of cells and the fermentation rate. Both studies found that at 35 ◦C, the cells in suspension were reduced approximately 50% compared to 30 ◦C. Similarly, they found a higher final attenuation level when fermenting at 35 ◦C compared to 30 ◦C. Torrija et al. [23] reported a drop in the ethanol yield of 11% approximately when fermenting at 35 ◦C. This low yield of ethanol was almost certainty linked to the reduced consumption of substrate.

**Figure 3.** Sugar concentration and fermentation rate calculated by taking the 1st derivative of the sugar consumption model (Equation (2) and Equation (1), respectively). Data shown represent fermentation data at 30 ◦C (brown circles) and 35 ◦C (purple squares); sugar consumption model at 30 ◦C (purple line) and 35 ◦C (brown line); and fermentation rate model at 30 ◦C (orange dashed line) and 35 ◦C (blue dashed line). Fermentations were performed in duplicate. The error bars represent the standard deviation of the data.

**Figure 4.** Viable cells in suspension throughout the fermentation. Data shown represent fermentation data at 30 ◦C (green circles) and 35 ◦C (red squares); cells in suspension model (Equation (3)) at 30 ◦C (green line) and 35 ◦C (red line). Fermentations were performed in duplicate. The error bars represent the standard deviation of the data.

### *3.2. Manual Image Analysis*

During the fermentation at both temperatures, changes in yeast size, as well as in the size of the vacuoles present were observed. Figures 5 and 6 show images of yeast cells and a summary of the average of the yeast and vacuole CSA throughout the fermentation for 30 ◦C and 35 ◦C, respectively. These values were manually assessed from 80–360 cells per sampling time using the methods described above. Images at 30 ◦C show that the average yeast CSA decreased from 43.4 µm2 ± 13.6 µm2 to 37.1 µm2 ± 11.9 µm2 , whilst at 35 ◦C there was a slight increase of CSA until after the fermentation peak (approximately after 15 h). Images also indicate a constant decrease in the size of vacuoles, until almost no vacuoles were present in the yeast cells towards the end of the fermentation at both temperatures. Given the large standard deviation in the size of the yeast, it was necessary to assess a large number of cells to provide statistically significant results. The manual analysis was labor-intensive, thus, substantial resources would be necessary to observe trends over the fermentation. To accomplish this goal, the automated analysis tool was developed so that many sampling times could be automatically assessed within a reasonable amount of time. Samples at the beginning, middle, and end of the fermentation processes were manually counted to validate the automated results.

**Figure 5.** Bright-field microscopic image of yeast cells (1000× magnification) of the duplicate runs from beginning, middle and end of the fermentation processes at 30 ◦C. The media included 0.01% *w/v* methylene blue solution to differentiate non-viable cells from the analysis. The yeast CSA and CSA ratio (vacuole to yeast area) were determined manually using the Nikon software.

**Figure 6.** Bright-field microscopic image of yeast cells (1000× magnification) of the duplicate runs from beginning, middle and end of the fermentation processes at 35 ◦C. The media included 0.01% *w/v* methylene blue solution to differentiate non-viable cells from the analysis. The yeast CSA and CSA ratio (vacuole to yeast area) were determined manually using the Nikon software.

### *3.3. Automated Image Analysis*

For this comparison, 80–360 yeast cells were automatically analyzed at each sampling period. To assess the accuracy of the automated analysis, the results for yeast CSA, total yeast CSA, and vacuoles CSA obtained through both automated and manual image analysis are compared in Table 2. The data obtained for yeast CSA and the CSA ratio vacuole to yeast, using both the automated image analysis and the manual Nikon software, resulted in an average difference of less than 4% and 2%, respectively. This shows that the data obtained in this study are reliable and comparable to the manual assessment where the Nikon software was used to measure cell morphology under the microscope.

### *3.4. Changes in Morphological State over Fermentation*

At 30 ◦C, there was a significant difference (*p* < 0.05) between the initial and the final yeast CSA at 30, whilst at 35 ◦C, there was not a significant difference (*p* > 0.05) between the initial and the final yeast CSA. Results regarding the CSA ratio between vacuole and yeast CSA showed a significant difference (*p* < 0.05) at 30 ◦C but not at 35 ◦C. Table 3 shows the average of the results obtained from the morphological analysis of yeast at each temperature.

**Table 3.** Average and standard deviation of yeast CSA and Ratio (vacuole to yeast area) summary \*.

\* The letters presented next to the standard deviation at each temperature were obtained through grouping averages based on Tukey's analysis. Different letters among each column represent significant difference.

The trend of yeast CSA for both temperatures did not show a significant change in size until the end of the fermentation (Figure 7). However, yeast at 35 ◦C had on average larger CSAs compared to yeast at 30 ◦C. These results are consistent with the finding of Gervais et al. [25] who found similar results of an increased cellular volume with increased temperature. They proposed that the increase of the cellular volume is due to a cellular response of yeast to heat shock.

**Figure 7.** (**a**) Yeast CSA with respect to time at 30 ◦C; (**b**) Yeast CSA with respect to time at 35 ◦C. Cross-sectional data are represented as follows: data at 30 ◦C (empty circles) and 35 ◦C (empty squares); average data at 30 ◦C (filled circles) and 35 ◦C (filled squares). Fermentations and automated image analysis were performed in duplicate. The error bars correspond to the standard error of the data, used to represent the high confidence in the mean associated with a large number of samples.

The ratio between the total CSA of the vacuoles and the total CSA of the cell (Figure 8) had a decreasing trend for 30 ◦C, but not for 35 ◦C (Table 3). This indicates that the vacuolar

space within the yeast cells was temperature-dependent. The changes undergone by the yeast show morphological changes (vacuole size) that were reported by Pratt et al. [6,7] and Izawa et al. [8]. However, the trends did not match exactly what was described in these studies. This is likely influenced by the highly stressful nature of rum fermentations. By quantifying the differences in yeast morphological state, it becomes possible to determine the impact of stress factors, and the quality of yeast for subsequent fermentations.

**Figure 8.** Vacuole and yeast CSA ratio with respect time at 30 ◦C (**a**) and 35 ◦C (**b**). Ratio data are represented as follows: data at 30 ◦C (empty circles) and 35 ◦C (empty squares); average data at 30 ◦C (filled circles) and 35 ◦C (filled squares). Fermentations and automated image analysis were performed in duplicate. The error bars represent the standard error of the data (used to represent the high confidence in the mean associated with a large number of samples).

The change in the yeast CSA, as well as the ratio between the size of vacuoles and the cell throughout the fermentation process, was obtained using an automated yeast analysis. Figures 7 and 8 clearly indicate that yeast has the capacity to undergo large morphological changes over a single fermentation, as has been observed by many researchers. Therefore, characterizing the rate of fermentation with respect to temperature, based upon the rate of sugar consumption per cell, is an incomplete assessment and it could lead to incorrect conclusions. A more detailed analysis was possible through the combination of the yeast CSAs obtained from the automated tool developed in this study, and the enumeration of viable yeast cells within suspension. Using the modeled number of cells in suspension and combining with the average surface area of yeast within suspension (estimated based on the assumption of yeast spherical shape), the total surface area of yeast in suspension over time was determined, as shown in Figure 9. This analysis has implications for reactions that are dependent upon cell wall area, such as nutrient uptake and stress responses.

Building upon this analysis, it became apparent that the rate of fermentation per cell changed over the duration of the fermentation and was dependent upon temperature. There are many factors that influence the rate of fermentation per cell, including temperature, sugar concentrations within the media, stress levels, etc. However, one parameter that could be accounted for was the total volume of yeast cells present. To determine the sugar consumption rate per volume of yeast, the authors used the CSA to determine the average volume of cells at each time period (assuming spherical cells). As the vacuole volume is not expected to be involved in cellular metabolism (and makes up a significant portion of the volume in early fermentation cells (Figure 8)), the vacuolar volume was subtracted from the average yeast volume calculation, resulting in an estimate of non-vacuolar cell volume. This value was multiplied by the model of cells in suspension to find the non-vacuolar cell volume per milliliter. Finally, the rate of sugar consumption model was divided by this value to find the sugar consumption rate per non-vacuolar cell volume; it is presented in Figure 10. This analysis shows that the sugar consumption rate per total non-vacuolar yeast volume declined with respect to time (and sugar concentration), and was higher at 35 ◦C. This analysis has applications for optimizing the reaction rate for fixed-bed,

encapsulated yeast and other fermentations where there is not expected to be new yeast growth. It can also help predict nutrient requirements and properties of yeast populations produced under different temperature profiles for alternative fermentation processes such as chemostats reactors.

**Figure 9.** Total surface area of yeast in suspension per milliliter with respect to fermentation time. Data are presented as follows: average total surface area of suspended yeast per milliliter at 30 ◦C (circles) and 35 ◦C (squares).

**Figure 10.** Fermentation rate per total non-vacuolar volume over time. Data are presented as follows: fermentation rate per total non-vacuolar volume at 30 ◦C (circles) and 35 ◦C (squares).

### **4. Conclusions**

Fermentations were conducted in duplicate at two different temperatures resulting in differences in yeast morphology and fermentation kinetics between the experiments. Standard methods were used to evaluate the fermentation properties, while changes to morphology were assessed using manual and automated image analysis. Differences in fermentation rate, number of cells in suspension, and cell size found were consistent with previous work. An average discrepancy of only 2–4% was found between the manual and automated analysis of the images. There were high standard deviations for cellular morphology measurements (due to natural variability present within the yeast population),

however, through the automated analysis of hundreds of cells, clear trends were observed. From the automated image analysis, changes to yeast CSA appeared to be temperaturedependent and progressed throughout the fermentation. Decreases in CSA ratio, defined as the ratio between the vacuole and yeast CSA, were observed at 30 ◦C and not at 35 ◦C. There were very small, or no vacuoles present in the cells at the end of the fermentation at 30 ◦C. Finally, the automated image analysis of cell morphology was combined with traditional measurements to provide insight into cellular surface areas and sugar consumption rates. The rate of fermentation per total non-vacuolar yeast volume decreased over the course of the fermentation and was higher at 35 ◦C despite a lower total rate of fermentation. This study showed that automated analysis of images can be used to track morphological changes in yeast over a fermentation, and that the combination of this analysis with traditional methods can be used to provide additional insight into yeast properties and behavior.

**Author Contributions:** Conceptualization, A.J.M. and M.G.-D.; methodology, M.G.-D.; software, M.C.; validation, M.G.-D., M.C.; formal analysis, A.J.M., M.G.-D.; investigation, M.G.-D.; resources, A.J.M.; data curation, M.G.-D., M.C.; writing—original draft preparation, A.J.M., M.G.-D., M.C.; writing—review and editing, K.A.T.-W.; visualization, M.G.-D., K.A.T.-W.; supervision, A.J.M.; project administration, A.J.M.; funding acquisition, A.J.M. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** Most of the data presented in this study is available within the article itself. The (large) collection of raw image files are available upon request from the corresponding author.

**Acknowledgments:** The authors would like to acknowledge Lallemand Inc. for the donation of yeast used throughout this research, the Food Science and Human Nutrition department at the University of Florida for use of the facilities, and Victor Cedeño for the technical support involving microscope and software.

**Conflicts of Interest:** The authors declare no conflict of interest.

### **References**

### *Article* **LC-ESI-QTOF-MS/MS Characterisation of Phenolics in Herbal Tea Infusion and Their Antioxidant Potential**

**Osbert Chou 1 , Akhtar Ali 1 , Vigasini Subbiah 1 , Colin J. Barrow 2 , Frank R. Dunshea 1,3 and Hafiz A. R. Suleria 1,2,\***

**Abstract:** Ginger (*Zingiber officinale* R.), lemon (*Citrus limon* L.) and mint (*Mentha sp.*) are commonly consumed medicinal plants that have been of interest due to their health benefits and purported antioxidant capacities. This study was conducted on the premise that no previous study has been performed to elucidate the antioxidant and phenolic profile of the ginger, lemon and mint herbal tea infusion (GLMT). The aim of the study was to investigate and characterise the phenolic contents of ginger, lemon, mint and GLMT, as well as determine their antioxidant potential. Mint recorded the highest total phenolic content, TPC (14.35 ± 0.19 mg gallic acid equivalent/g) and 2,20 -azinobis(3-e-thylbenzothiazoline-6-sulfonic acid), ABTS (24.25 ± 2.18 mg ascorbic acid equivalent/g) antioxidant activity. GLMT recorded the highest antioxidant activity in the reducing power assay, RPA (1.01 ± 0.04 mg ascorbic acid equivalent/g) and hydroxyl radical scavenging assay, •OH-RSA (0.77 ± 0.08 mg ascorbic acid equivalent/g). Correlation analysis showed that phenolic content positively correlated with the antioxidant activity. Venn diagram analysis revealed that mint contained a high proportion of exclusive phenolic compounds. Liquid chromatography coupled with electrospray ionisation and quadrupole time of flight tandem mass spectrometry (LC-ESI-QTOF-MS/MS) characterised a total of 73 phenolic compounds, out of which 11, 31 and 49 were found in ginger, lemon and mint respectively. These characterised phenolic compounds include phenolic acids (24), flavonoids (35), other phenolic compounds (9), lignans (4) and stilbene (1). High-performance liquid chromatography photometric diode array (HPLC-PDA) quantification showed that GLMT does contain a relatively high concentration of phenolic compounds. This study presented the phenolic profile and antioxidant potential of GLMT and its ingredients, which may increase the confidence in developing GLMT into functional food products or nutraceuticals.

**Keywords:** polyphenols; LC-ESI-QTOF-MS/MS; HPLC; medicinal plants; ginger; lemon; mint; herbal tea infusion; antioxidants

### **1. Introduction**

Herbal teas made from various medicinal plants are rich sources of phenolic compounds and are consumed by many cultures all around the world. Medicinal plants prepared for tea infusions have been studied for their antioxidant, anti-inflammatory and other properties [1]. Herbal teas have also been reported to exhibit synergistic antioxidant effects, which increases their value as beverages for potential health benefits [2,3]. Common medicinal plants such as ginger, lemon and mint can be fused together as ginger lemon mint tea (GLMT) and be consumed as herbal tea to improve health being.

Gingers (*Zingiber officinale* Roscoe) from the family *Zingiberaceae* [4–6], are native to Southeast Asia and have been incorporated into a diverse array of cuisines. In China and

**Citation:** Chou, O.; Ali, A.; Subbiah, V.; Barrow, C.J.; Dunshea, F.R.; Suleria, H.A.R. LC-ESI-QTOF-MS/MS Characterisation of Phenolics in Herbal Tea Infusion and Their Antioxidant Potential. *Fermentation* **2021**, *7*, 73. https://doi.org/10.3390/ fermentation7020073

Academic Editor: Daniel Cozzolino

Received: 4 April 2021 Accepted: 6 May 2021 Published: 10 May 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

India, gingers are regarded as medicine [5]. Ginger is reported to have various health benefits as it demonstrates antioxidant, antimicrobial and anti-inflammatory activities [7]. A gingerol is a group of phenols commonly found in the rhizomes of *Zingiberaceae* plants, which have been extensively studied for their cytotoxic potential [8]. Lemon (*Citrus limon* L.) is from the citrus family *Rutaceae* [9], and has been demonstrated to contain phenolic compounds which are beneficial to health [10]. Eriocitrin, a phenolic compound in lemons was found to increase antioxidant activities in plasma after ingestion [11]. Lemon has potent antioxidant activities in its peel and zest and its protective properties against DNA and cell damage have prompted the research on lemon as a cancer preventative [12]. Other studies have also shown that lemon's essential oil has antioxidant and antimicrobial effects [13]. Mint (*Mentha sp.*) is from the *Lamiaceae* plant family [14,15]. The essential oils of plants within the *Mentha* genus exhibit antioxidant and antimicrobial activities [16,17]. Notable antioxidants in mints were found to be rosmarinic acid, caffeic acid, α-terpinene, luteolin and eriocitrin and these compounds are believed to be the major antioxidants particularly found in peppermint [15].

Phenolic compounds are a large group of secondary plant metabolites commonly found in fruits, vegetables and spices, which are thought to play prominent roles in human nutrition and health [18]. Some of these metabolites produced by plants are classified as antioxidants [19]. Antioxidants are molecules that can reduce the damaging effects of free radicals, which are chemicals with one or more unpaired electrons produced in an organism via natural metabolism or environmental factors [20]. Free radicals can turn non-free radical molecules in the body into radicals, cascading a chain reaction that causes destructive effects within cells and tissues [21]. Thus, medicinal plants rich in antioxidants have been lauded and advised to be part of the human diet to promote health and wellbeing [22,23].