[1]Fabiana Ferracina

[1]Bala Krishnamoorthy These authors contributed equally to this work.

These authors contributed equally to this work.

These authors contributed equally to this work.

[3]Vidyasagar Sathuvalli These authors contributed equally to this work.

[1]Department of Mathematics and Statistics, Washington State University,
Vancouver, WA, USA

2]Pacific Northwest National Laboratory, Richland, WA, USA

3]College of Agricultural Sciences, Oregon State University, Hermiston, OR,
USA

# Predictive Analytics of Varieties of Russet Potatoes

0000-0003-4634-4880<EMAIL_ADDRESS>0000-0002-2727-6547
<EMAIL_ADDRESS>Mahantesh Halappanavar  0000-0002-2323-4753<EMAIL_ADDRESS>Shengwei Hu  0009-0003-7150-3617<EMAIL_ADDRESS>0000-0001-5727-9987
<EMAIL_ADDRESS>*  [  [

###### Abstract

We explore the application of machine learning algorithms to predict the
suitability of Russet potato clones for advancement in breeding trials.
Leveraging data from manually collected trials in the state of Oregon, we
investigate the potential of a wide variety of state-of-the-art binary
classification models. We conduct a comprehensive analysis of the dataset that
includes preprocessing, feature engineering, and imputation to address missing
values. We focus on several key metrics such as accuracy, F1-score, and
Matthews correlation coefficient (MCC) for model evaluation. The top-
performing models, namely the multi-layer perceptron (MLP), histogram-based
gradient boosting (HGB), and support vector machine (SVM) classifiers
demonstrate consistent and significant results. Variable selection further
enhances model performance and identifies influential features in predicting
trial outcomes. The findings emphasize the potential of machine learning in
streamlining the selection process for potato varieties, offering benefits
such as increased efficiency, substantial cost savings, and judicious resource
utilization. Our study contributes insights into precision agriculture and
showcases the relevance of advanced technologies for informed decision-making
in breeding programs.

###### keywords:

Russet potatoes, machine learning, classification, variable selection

##  1 Introduction

Potato (_Solanum tuberosum_ L.) is an important food crop in the United Sates
with an annual potato production of 392.24 million cwt in 2022. The Pacific
Northwest forms the largest sub-region in potato production in the U.S.,
contributing 60.1% of U.S. potato production according to USDA’s National
Agricultural Statistics Service. Potatoes are not only consumed as a fresh
vegetable but also form a raw material for the French fry, chip, and starch
processing industries. In the U.S., 39% of the potato crop is utilized by the
frozen processing sector. The Columbia basin of Oregon and Washington states
is the leader in the frozen potato sector, with more than 80% of potatoes
grown here being russet type for utilization by the processing industry.

Developing new russet potato varieties with high yield, good processing
quality, climate resilience, and disease resistance is the main focus of the
Oregon State University’s Potato Breeding and Variety Development program, an
integral part of the Northwest Variety Development Program (also known as the
Tri-State program). Typically, it takes 12–13 years from the hybridizations to
select and release a new potato variety. In the first field year, seedling
tubers generated from true potato seeds are planted in Klamath Falls (around
60,000 clones) as single hills and about 1% of the clones selected are planted
at Hermiston and Klamath Falls in the second field year. Phenotypic field
selections are performed in the first and second years and then the selected
clones are tested through Oregon statewide replicated trials at Hermiston,
Klamath Falls, and Ontario. The Oregon statewide trials are typically
performed in years three to five and the selected clones will then move to a
Tri-State trial, and then to regional trials from the sixth or seventh field
year to the tenth or eleventh year. Potato varieties released from the Tri-
State variety development program such as the “Clearwater Russet”, “Mountain
Gem Russet”, “Umatilla Russet”, and “Ranger Russet” have proved to be
significantly successful by producing higher total yield and higher percentage
of finished product derived from raw tubers in the processing industry.
Benefits from the adoption of new russet potato varieties continue to grow
over time [1].

In the breeding program, the selected potato clones must be grown and
evaluated in research replicated trials for multiple years. Each selected
clone will be planted in diverse locations and agronomic data is collected and
evaluated for its performance. Data for each potato clone collected annually
includes field data such as emergence, plant vigor, plant uniformity, stems
per plant, vine maturity; after harvest data includes total yield, yield for
different weight categories; attributes pertaining to the potato’s external
characteristics, specific gravity, internal defects, and performance following
the process of frying. The extensive costs associated with managing these
selection trials and the subsequent data collection after harvest account for
a significant proportion of investment in the Oregon Potato Variety
Development Program.

The complexity further increases due to the intricate interplay between
distinct traits and environmental interactions needed for multi year and multi
location trials. Breeders must decide whether the potato clone should go into
the subsequent selection trial stage based on its performance or whether
retaining a poorly performing clone could add unnecessary cost to the program
resulting in decreased breeding selection efficiency. Advances in genetic
tools have resulted in the use of molecular markers and genomic selection to
improve selection efficiency. Even with molecular and genomic data, it is
essential to carry multi year, multi location trials and the breeder must
select which clones to advance. In this context, artificial intelligence (AI)
or machine learning technologies can help evaluate potato clones and assist in
making such decisions [7]. The ascendancy of AI technologies in
revolutionizing data analytics is evident, particularly within breeding
programs. Researchers have used machine learning models to obtain highly
accurate predictions of root yield based on the vegetative indices of the
different growth stages of cassava (Manihot esculenta Crantz) [18]. With the
potent integration of machine learning algorithms and chemical composition as
predictors, researchers have successfully predicted consumer flavor
perceptions of blueberries and tomatoes [4]. Data-informed decision making
facilitated by machine learning models can assist breeders select potato
clones more accurately and reduce the number of varieties proceeding to
subsequent stages. The consequential impact includes substantial financial
savings, increased breeding selection efficiency, and judicious resource
utilization.

###  1.1 Related Work

Precision agriculture and the incorporation of advanced technologies like
artificial intelligence are becoming crucial for maintaining high-quality
yields for many crops. Machine learning methods have been applied to many
crops whose information was obtained through various data collection
techniques [6, 2].

In a study of French fry potato in Poland, machine learning methods of
regression to predict yield using diverse datasets (agronomic, weather,
satellite, and soil data) were used [11]. The authors found that consideration
of features, coupled with appropriate data preprocessing techniques (e.g.,
imputation) and outlier management, is crucial for accurate predictions in the
field of agricultural yield forecasting.

Another study of potato employed the machine learning models of support vector
machines (SVM) and multi-layer perceptrons (MLP) to classify potato varieties
based on macroscopic observations and measurements for selection or de-
selection during trials much like those used in our study [13]. For each of
these two classification techniques, four models were constructed—two for
binary classification and two for 4-class classification. The results were
reported in terms of the accuracy score (number of correct predictions) with
the SVM method demonstrating higher accuracy than MLP in the 4-class models.
It is unclear how many features [13] considered or how they addressed missing
data. Further, no parameter tuning was performed.

##  2 Methods

Code for the methods described here can be found in our repository:
https://github.com/fabstat/burbank.

###  2.1 Study Area and Data

From 2013–2014 there were four regions in Oregon, represented in Figure 1, in
which the Russet potato trials were conducted: Hermiston (HER), Ontario (ONT),
Klamath Falls (KF) and Corvallis (COR). Beginning in 2015, COR was dropped
from the russet trials and two trials were conducted in HER, early and late
(see Table 1).

Figure 1: US State of Oregon. Points represent the cities in which the potato trials were conducted: Corvallis, Hermiston, Ontario, and Klamath Falls. From each of these states, russet potatoes such as the one depicted at the center of the map (source: Wikimedia) were collected and data of each clone’s characteristics were recorded for analysis. Table 1: Early and late planting season dates over the years. | Early Planting | Late Planting
---|---|---
Year | Planting | Vine Kill | Planting | Vine Kill
2013 |  |  | 04/05/13 | 09/04/13
2014 |  |  | 04/16/14 | 09/11/14
2015 | 03/26/15 | 07/22/15 | 04/06/15 | 09/04/15
2016 | 03/25/16 | 07/29/16 | 04/07/16 | 09/02/16
2017 | 03/27/17 | 08/10/17 | 04/04/17 | 08/28/17
2018 | 03/26/18 | 08/08/18 | 04/09/18 | 08/31/18
2019 | 03/28/19 | 08/01/19 | 04/16/19 | 09/02/19
2020 | 03/23/20 | 08/01/20 | 04/21/20 | 08/31/20
2021 | 03/17/21 | 07/30/21 | 03/31/21 | 08/31/21
|  |  |  |

The preprocessed dataset includes 1086 total clones, 216 of which are unique.
Each clone is trialed 1–3 years. Most clones were in their first year of
trial. There are three control varieties planted in all regions and all years:
Ranger Russet, Russet Burbank, and Russet Norkotah. Two other control
varieties are tested only in Hermiston: Shepody (early) and Umatilla (late),
and were measured for some but not all years. For each clone, data for 38
attributes are recorded in a spreadsheet for each particular region. This data
pertained to the total yield, yield per size category, features related to the
potato’s appearance, and features related to the potato after being fried. See
Table A for more details on each feature.

###  2.2 Feature Engineering

Not all 38 aforementioned features were used in our analysis, and some
additional features were included. Features with over 400 not applicable (NA)
ratings were dropped: “flower color”, “vine size”, and “maturity”. The feature
“bruise data” was also dropped as it contained mostly text description and was
missing for most observations. The following new features were added:

  1. 1.

“true_keeps”: response variable. A clone is assigned a “1” if it passed three
trials and a “0” if it passed fewer than three trials. Clones in the first or
second year of the statewide trials during this analysis were removed from the
dataset.

  2. 2.

“Ctrl ave”: Average total yield of control clones for a particular year and
trial region. All clones from a particular year and trial region should have
the same “control average”.

  3. 3.

“% CA”: $\displaystyle 100\times\left(\frac{\textnormal{Total
Yield}}{\textnormal{Ctrl Ave}}\right)$ represents the percentage of a
particular variety’s total yield from the averaged controls’ total yield.

  4. 4.

“GDD 1–60”: the sum of the first 60 growing degree days

  5. 5.

“GDD 61–90”: the sum of growing degree days from day 61 to day 90

  6. 6.

“GDD 91–end”: the sum of growing degree days from day 91 to vine kill date

The last three variables pertain to _growing degree days_ (GDD) defined as

$\mathrm{GDD}=\left[\frac{(T_{\mathrm{MAX}}-T_{\mathrm{MIN}})}{2}\right]-T_{\mathrm{BASE}}$

where $T_{\mathrm{MAX}}$ and $T_{\mathrm{MIN}}$ are the day’s maximum and
minimum air temperatures, respectively, and $T_{\mathrm{BASE}}$ is “the
temperature below which the process of interest does not progress” [12]. In
our case, $T_{\mathrm{BASE}}$ is the lowest temperature at which potato
continues to grow, or 8∘ Celsius.

###  2.3 Data Imputation

While the overall dataset was reasonably large, a significant number of
variables were missing values (or entries) for several observations. For
instance, the variables “flower color” and “maturity” were missing for 638 and
639 observations, respectively (of the total of 1086 clones). When considering
only observations with numeric yield, 25 of the 36 original attributes have at
least 1 missing missing value, with 13 of these features each having more than
100 missing values. Hence, imputation was necessary to consider the complete
dataset. Without imputation, one must drop observations with missing values.
Since a few of the attributes were not recorded for almost half of the
dataset, dropping observations after data processing and cleaning would reduce
the set from 1086 observation to 209 observations. Such a small set of
observations would lead to overfitting and machine learning that might not be
meaningful.

To impute data, we first standardized the dataset and then used Scikit-learn’s
[14] function `IterativeImputer` with `sample_posterior=True`, which performs
multiple imputation by chained equations, or MICE [20]. Under the assumption
that the data is missing at random, MICE works by first performing a simple
mean imputation to serve as place holders for all variables. For each variable
in turn, the place holder means are removed and the variable is treated as
dependent on the other independent variables in a regression problem.

Besides the imputed dataset, we also constructed a non-imputed dataset where
we chose a maximum number of “NA’s” any given column could have so that the
dataset was reduced to at most the largest number of NA’s that was below this
maximum threshold. In the analysis presented below, an NA threshold of 400 was
selected to ensure we kept at least two thirds of the dataset even when
dropping rows containing NA’s. For purposes of comparison, we constructed both
the imputed and non-imputed datasets by first dropping columns with over 400
NA’s. Postprocessing there were 885 observations in the imputed set and 404
observations in the non-imputed set, with the same set of 40 features in each.

Once numerical variables were standardized, we hot encoded the categorical
variables - a process where all categorical variables’ values are accounted
for all observations, but for an observation’s given category, its value is
encoded to 1 while the other non-applicable values are encoded to zero - and
handled missing values in a similar manner, i.e., they were either dropped or
imputed. We then ran a grid search with cross validation to find the best set
of hyperparameters for each of the models considered (see Section 2.5).

###  2.4 Modeling Approach

With the ultimate goal of automating the process of discriminating varieties
that meet or exceed expected targets from those that do not meet the standards
specified above, we explored a large family of machine learning classification
models on our dataset and drew structural insights from their results. Note
that this process is currently fully manual, where the potato breeding expert
looks at every datum in order to make a decision. As such, having a more
automated way to make, or aid in making, the decision is of great advantage.
We concentrated on nonlinear classifiers since we expected them to perform as
well or better than linear ones. Nonlinear classifiers are more flexible. They
do not rely on a line, or hyperplane in higher dimensions, to separate the
data classes, and they are more robust against outliers.

Clones must be selected for breeding in all three years of the Oregon
statewide trials, if they are to graduate to the Tri-state trials. We labeled
each clone in the dataset as “1” if it passed three trials and as “0” if it
passed fewer trials. Clones were removed from the dataset if still in the
first or second year of statewide trialing during this analysis. We performed
model selection and hyperparameter tuning via grid searches using 5-fold
cross-validation and split the data such that 80% was used in training and 20%
was used for testing.

###  2.5 Classification Models and Evaluation Metrics

Since the goal was to separate two classes in a high dimensional feature space
efficiently, we constructed a large family of nonlinear classification models.
As these classification methods may not be widely used for agricultural data
analysis, we present high level explanations for each of them below.

  1. 1.

$\boldsymbol{k}$-Nearest neighbors: This is a fundamental and widely applied
clustering method in unsupervised learning. The objective of the algorithm is
to partition a dataset into $k$ clusters (for a predefined value of $k$) based
on similarity, by iteratively computing the Euclidean distance of each data
point and cluster representatives, assigning the data points to clusters, and
then moving the cluster representative to be the center of the data points it
represents. Such an algorithm leads to a local optimum and so it is sensitive
to initial conditions and shape assumptions. Also, since clustering relies on
computing arithmetic means, this algorithm is sensitive to outliers [17].

  2. 2.

Gaussian process based on Laplace approximation: This approach involves
placing a Gaussian Process (GP) prior over a latent function, which is then
transformed through the logistic function to obtain a prior on the class
probability. The Laplace approximation is employed for posterior inference,
approximating the non-Gaussian joint posterior with a Gaussian distribution.
With this method, positive training examples will give rise to a positive
coefficient for their kernel function, while negative examples will give rise
to a negative coefficient, making it analogous to the support vector machine
(SVM) model (see Method 10 below). However, all features provide information
to perform predictions, making it an inefficient algorithm in high dimensional
spaces [21].

  3. 3.

Gaussian naïve Bayes: This is a naïve Bayes method where the likelihood of the
features is assumed to be Gaussian. A naïve Bayes method is a supervised
learning technique grounded in the application of Bayes’ theorem, which
incorporates the “naïve” assumption of conditional independence among all
pairs of features given the class variable $y$ and features
$x_{1},\dots,x_{n}$ as follows:

$\displaystyle
P(y|x_{1},\dots,x_{n})=\frac{P(y)P(x_{1},\dots,x_{n}|y)}{P(x_{1},\dots,x_{n})}$
$\displaystyle\Rightarrow$ $\displaystyle P(y|x_{1},\dots,x_{n})\propto
P(y)\prod\limits_{i=1}^{n}P(x_{i}|y).$

The predicted class variable maximizes the right-hand side of the proportion,
given the relative frequency of the class variable in the training dataset
[22].

  4. 4.

Quadratic discriminant analysis: This supervised learning technique computes
discriminant functions for each class, which are then used to determine the
decision boundary between each class. Assuming each class density has a
multivariate Gaussian distribution and by comparing the log-ratio of two
classes posteriors, we can obtain the quadratic discriminant functions

$\delta_{k}(x)=-\frac{1}{2}\log|\boldsymbol{\Sigma}_{k}|-\frac{1}{2}(x-\mu_{k})^{T}\boldsymbol{\Sigma}_{k}^{-1}(x-\mu_{k})+\log\pi_{k}\,.$

The decision boundary between each pair of classes $k$ and $l$ is the set of
$x$ values satisfying the quadratic equation $\delta_{k}(x)=\delta_{l}(x)$
[9].

  5. 5.

Decision tree: This is a supervised learning approach in which a given vector
of explanatory variables $X$ is partitioned into the nodes of a binary tree
whose edges represent the mutually exclusive set of outcomes of the parent
node. Each edge leads to a different input feature until it reaches the leaf
nodes, where a class is assigned to the sequence of feature node outcomes; the
goal is to group samples with the same label. Classification depends on the
underlying decision tree structure, which is decided by hyperparameters such
as the split criteria and the maximum depth of the tree. Although simple to
understand, decision trees can be unstable or cause over-fitting depending on
the choice of hyperparameters [9].

  6. 6.

Adaptive boosting: This is a commonly used ensemble learning approach in which
a sequence of weak learners (models that are slightly better than guessing
randomly), such as small decision trees, is repeatedly applied to modified
versions of the data (i.e., by changing the weights applied to the training
observations). The predictions from all the weak classifiers are then combined
to produce a majority vote prediction [9].

  7. 7.

Random forest: This ensemble learning approach is an extension of the
bootstrap aggregating (or bagging) method, in which random samples of the
training data are selected with replacement and then trained on individually
using the base learner. Random forests work similarly by constructing an
uncorrelated collection of decision trees. Each tree in the ensemble is
constructed from training data sampled with replacement, and an exhaustive
search through the features is performed in order to obtain the best split in
the node set. Although this is a robust algorithm and might be selected for
this reason, it is more complex and time consuming than other ensemble
techniques [3].

  8. 8.

Multi-layer perceptron (MLP): This is a feedforward neural network with at
least three layers—an input layer, one or more hidden layers, and an output
layer—in which neurons are fully connected with a nonlinear activation
function. The MLP classifier is trained by backpropagation to optimize a loss
function. The Broyden–Fletcher–Goldfarb–Shanno (BFGS) method, the optimization
algorithm used in this framework, is an iterative method for solving
unconstrained nonlinear optimization problems [8].

  9. 9.

Histogram-based gradient boosting (HGB): Gradient boosting algorithms are
additive machine learning models that combine many weak learners such as
decision trees to form a strong learner. As finding the best split points for
learning the decision trees via sorting the values of each feature is time and
memory intensive, the histogram based algorithm buckets continuous feature
values into discrete bins and constructs a histogram, thus reducing the number
of unique values to sort [10]. Several hyperparameters in the histogram-based
gradient boosting classifier (HGBC) can be optimized during learning, such as
maximum tree depths, maximum number of bins and the $L_{2}$ regularization
parameter, to avoid over-fitting issues while affording good performance.

  10. 10.

Support vector machine (SVM) with a radial basis function kernel: A linear
support vector machine (SVM) identifies the plane that maximizes the distance
between itself and the closest set of points (specifically the feature vectors
from a set of observations) that belong to the two distinct categories. Figure
2(a) shows a linearly separable dataset with a linear SVM decision boundary.

(a) Two classes of points separated by a line.

(b) Two classes of points separated by a circular curve.

Figure 2: A dataset where the two classes of points are separable with a line
(Left), and one where a nonlinear kernel (radial basis function or RBF in this
case) does the separation (Right) when no linear separation is possible.

If the data is not linearly separable for the desired classification problem,
then there will be no hyperplane separating the two classes. However, a
nonlinear decision boundary may still exist. As SVM is an inherently linear
classification method, we transform the vector space via a _kernel trick_. A
kernel is a function that take as inputs the observations of the original
feature space and outputs the dot products of vectors in a transformed target
space where finding hyperplanes that separate the data is more likely [16].
Two important hyperparameters in a nonlinear SVM are $C$ and $\gamma$. The
scale parameter $\gamma$ determines the radius of influence of a single
observation, while $C$ controls the size of slack variables that allow some
observations to be outside the target hyperspace. Setting these parameters
appropriately is a trade-off between controlling the classification error and
overfitting the data.

  11. 11.

An ensemble of classifiers along with a meta-classifier: Also called a
stacking classifier, two or more classifiers are applied to the datasets and
the resulting classification predictions are used to train a meta-classifier
[19]. The stacking classifier used in this study has gradient boosting and SVM
as its level-1 classifiers, and logistic regression as its level-2 (or meta-)
classifier.

####  2.5.1 Evaluation Metrics

To evaluate the quality and performance of the large family of classification
models, we computed several metrics including accuracy (proportion of correct
predictions), F1-score (harmonic mean of precision and recall), and Matthews
correlation coefficient (MCC), which is defined as:

MCC
$\displaystyle=\frac{\text{TP}\times\text{TN}~{}-~{}\text{FP}\times\text{FN}}{\sqrt{(\text{TP}+\text{FP})(\text{TP}+\text{FN})(\text{TN}+\text{FP})(\text{TN}+\text{FN})}}\,,$
(1)

where TP is the number of true positives, TN the number of true negatives, FP
the number of false positives, and FN the number of false negatives. The MCC
ranges from $-1$ to $1$ and on imbalanced datasets, it can be more informative
than either accuracy or the F1-score [5]. Our imbalanced dataset includes 88
positive examples and 998 negative examples.

####  2.5.2 Variable Selection

We performed variable selection on both the imputed and non-imputed datasets
for the top-performing classification models used in this study. For each
model described in Section 3.1, we performed a grid search with cross-
validation on the hyperparameter space to determine the best scoring model. We
then performed forward feature selection by beginning with zero features in
our models and iterating through each available feature, computing each
model’s score with that feature included in it. The feature resulting in the
highest model score becomes a feature in the model, and the search for a
second feature proceeds. The process continues until we have the desired
number of features in the model or an additional feature does not
significantly improve the model score. The results from variable selection are
presented in Section 3.1.

##  3 Results

Table 2 presents the MCCs for the classification models explored in this study
on the imputed and non-imputed datasets. Columns containing more than 400 NA’s
were dropped before imputation and data cleaning, so that both the imputed and
non-imputed datasets contained the same set of features. The top performing
classifiers (highlighted in gray) were the multi-layer perceptron (MLPC),
histogram-based gradient boosting (HGBC), and support vector machine (SVC), as
each obtained an MCC score above or very close to 0.5 in the cross-validation
and test for both imputed and non-imputed datasets with high significance. We
note that AdaBoosting achieved similar performance to the declared top
performing models, however we will not discuss it further as we will focus on
HGBC, which uses the same underlying principle of an ensemble of weak learners
but it is a more modern algorithm allowing additional hyperparameters to be
set (e.g., regularization).

Models whose test MCC is very close to zero are no better than a random guess.
Gaussian process, Naïve Bayes, QDA, and Decision tree classifiers all
performed particularly poorly on our datasets. Although Gaussian process is
analogous to SVM, its implementation used in this analysis is non-sparse, and
hence the model uses all features for each example when computing the high-
dimensional posterior. As such, a possible explanation for this model’s poor
performance is that the posterior computed from the training examples is
skewed. If the posterior is skewed, the Laplace approximation (which is
centered on the mode) will not be accurate [15].

The poor performance from Naïve Bayes and QDA may be due to the assumption of
feature independence inherent to these models. An inspection of the table of
features (Table A) makes it clear that several variables considered are not
independent. Lastly, a decision tree’s performance is very sensitive to its
structure, where features are greedily made into nodes. It is possible that
variable selection and a more thorough search of the hyperparameter space
could allow these models to better perform on our dataset. However, here we
focus on those models showing better promise under the same circumstances.

Table 2: Best MCCs for all the classification models considered in this study. Top performing models (highlighted in gray) were multi-layer perceptron (MLPC), histogram-based gradient boosting (HGBC) and support vector machine (SVC), on datasets with both multiple imputation and without imputation. The results presented here are from models using all available features which have less than 400 NA’s. | Imputed | Not Imputed
---|---|---
Model | Test MCC | CI Test MCC | Test MCC | CI Test MCC
$k$-NN | 0.413 | (0.378, 0.447) | 0.338 | (0.283, 0.392)
Gaussian process | 0.000 | ($-$0.042, 0.042) | 0.000 | ($-$0.062, 0.062)
Naïve Bayes | 0.373 | (0.336, 0.408) |  $-$0.004 | ($-$0.066, 0.058)
QDA | 0.288 | (0.250, 0.326) | 0.000 | ($-$0.062, 0.062)
Decision tree |  $-$0.035 | ($-$0.077, 0.006) | 0.089 | (0.027, 0.150)
AdaBoosting | 0.523 | (0.492, 0.553) | 0.541 | (0.496, 0.583)
Random forest | 0.085 | (0.043, 0.126) | 0.531 | (0.486, 0.574)
MLPC | 0.530 | (0.500, 0.559) | 0.569 | (0.525, 0.609)
HGBC | 0.563 | (0.534, 0.591) | 0.485 | (0.436, 0.531)
SVM | 0.563 | (0.534, 0.591) | 0.692 | (0.658, 0.723)
Stack | 0.534 | (0.504, 0.563) | 0.432 | (0.380, 0.481)
|  |  |  |

###  3.1 Top Performing Models

From among the large family of classification models considered (Section 2.5),
three models performed consistently well with both imputed and and non-imputed
data. They were the multi-layer perceptron classifier (MLPC), the histogram-
based gradient boosting classifier (HGBC), and the support vector machine
classifier (SVC). The results from these models without variable selection are
highlighted in Table 2 in light gray, and their confusion matrices are
presented in Figure 3. We explored forward feature selection on these three
models to understand which variables are important in determining whether a
potato clone is kept in the trials. The resulting MCC scores for the top three
performing models using forward feature selection on the imputed and non-
imputed datasets are presented in Table 3.

Figure 3: Confusion matrices for the top performing models applied to both imputed and non-imputed datasets: multi-layer perceptron classifier (MLPC, called a Neural Net here) in first row, histogram-based gradient boosting classifier (HGBC) in middle row, and support vector machine (SVM) classifier in the last row. Variable selection was not performed here, however variables containing at least 400 NA’s were dropped before training. Numerical values within the confusion matrices represent prediction proportions. Table 3: MCC for the top classification models after forward feature selection.  | Imputed | Not Imputed
---|---|---
Model |  $\Delta$CV MCC | Test MCC | CI Test MCC |  $\Delta$CV MCC | Test MCC | CI Test MCC
MLPC | 0.476 | 0.608 | (0.581, 0.633) | 0.595 | 0.623 | (0.584, 0.659)
HGBC | 0.531 | 0.574 | (0.546, 0.602) | 0.511 | 0.645 | (0.608, 0.680)
SVM | 0.539 | 0.502 | (0.471, 0.533) | 0.731 | 0.623 | (0.584, 0.659)
|  |  |  |  |  |

It is interesting to note that variable selection significantly improved the
test MCC of MLPC on both imputed and non-imputed datasets, as well as the test
MCC of HGBC on the non-imputed dataset. However, this form of variable
selection seems to hinder the performance of the SVM classifier on the imputed
and non-imputed datasets. One reason for this drop in performance could be
that features were selected after the hyperparameter search. Both techniques
use the SVM classifier, which in turn suppresses certain features through an
$\ell_{2}$-norm penalty. Thus, the second application of SVM for feature
selection would drop the variable with high penalty regardless of their
overall contribution to the decision function.

Table 4 lists the variables that were selected for each of the three top-
performing models on each imputed and non-imputed dataset. We highlight in
bold the variables selected in all three models when applied to a particular
dataset. On the imputed dataset the variables “Trial Region_COR”, “Trial
Region_HER_Early”, “Yr in trial”, “%RB”, “Vascular”, “GDD 61–90” and “GDD
91–end” were selected for all three models. On the other hand, the common
variables selected for the non-imputed dataset were “Trial Region_HER_Late”,
“Yr in trial”, “Hollow Heart”, “Brown Center”, ‘Vascular”, “Shape Uniformity”,
and “GDD 1–60” instead.

Two variables selected in the models using the imputed dataset, “Trial
Region_COR” and “Trial Region_HER”, gained their significance due to only
being present for the years 2013 and 2014. Respectively, they accounted for
4.9% and 5.2% of the total kept varieties for all regions and years present in
the imputed dataset. This imbalance may be the sole reason these variables
were selected as informative.

The variables selected by all three models and for both datasets were “Yr in
Trial” and “Vascular”. The importance of “Yr in Trial” may be irrelevant as it
indicates how long a particular potato variety survived selection during the
three-year trial. “Vascular” refers to vascular discoloration, a darkening of
the ring of tissue that surrounds the tuber just below its skin. The ring
contains the tuber’s vascular system, and its discoloration can be indicative
of bacterial or fungal infections or abiotic stress response. The vascular
discoloration in tubers can lead to low processing quality and is an important
trait for selection.

Table 4: Variables selected via forward feature selection (second column) using the corresponding top performing algorithm on each dataset (first column). Bold variable names highlight the variables that were selected in common between the algorithms when applied to the same dataset (imputed versus not imputed). Model |  Variables Selected
---|---
MLPC (imputed dataset) |  “Trial Region_COR”, “Trial Region_HER_Early”, “Trial Region_ONT”, “Yr in trial”, “Total Yield”, “%RB”, “%RB 1s”, “%1s”, “4-6oz”, “Y2s + Over 20oz”, “Culls”, “Specific Gravity”, “Blackspot”, “IBS”, “Vascular”, “Tuber Shape”, “Eye Depth”, “Ctrl Ave”, “% CA”, “GDD 1–60”, “GDD 61–90”, “GDD 91–end”
HGBC (imputed dataset) |  “Trial Region_COR”, “Trial Region_HER”, “Trial Region_HER_Early”, “Trial Region_HER_Late”, “Trial Region_KF”, “Trial Region_ONT”, “Yr in trial”, “%RB”, “Over 20oz”, “4-6oz”, “Under 4oz”, “Culls”, “Tuber/Plant”, “Sugar Ends”, “Hollow Heart”, “Brown Center”, “IBS”, “Vascular”, “Russeting”, “Ctrl Ave”, “GDD 61–90”, “GDD 91–end”
SVM (imputed dataset) |  “Trial Region_COR”, “Trial Region_HER”, “Trial Region_HER_Early”, “Trial Region_HER_Late”, “Trial Region_KF”, “Trial Region_ONT”, “Yr in trial”, “Total Yield”, “%RB’’, “Y1s”, “%RB 1s”, “%1s”, “Y2s + Over 20oz”, “Specific Gravity”, “Fry Color Stem”, “Fry Color Bud”, “Sugar Ends”, “Blackspot”, “Vascular”, “Scab”, “GDD 61–90”, “GDD 91–end”
MLPC (non-imputed dataset) |  ‘Trial Region_HER_Early”, “Trial Region_HER_Late”, “Trial Region_ONT”, “Yr in trial”, “%RB 1s”, “10-20oz”, “6-10oz”, “4-6oz”, “Length/Width”, “Specific Gravity”, “Fry Color Stem”, “Hollow Heart”, “Brown Center”, “Vascular”, “Tuber Shape”, “Shape Uniformity”, “Growth Cracks”, “Ctrl Ave”, “GDD 1–60”, “GDD 61–90”, “GDD 91–end”
HGBC (non-imputed dataset) |  “Trial Region_HER_Early”, “Trial Region_HER_Late”, “Trial Region_ONT”, “Yr in trial”, “Total Yield”, “%RB”, “Y1s”, “%RB 1s”, “Over 20oz”, “10-20oz”, “Culls”, “Tuber/Plant”, “Hollow Heart”, “Brown Center”, “Blackspot”, “Vascular”, “Shape Uniformity”, “Eye Depth”, “Scab”, “Shatter Bruise”, “GDD 1–60”
SVM (non-imputed dataset) |  “Trial Region_HER_Late”, “Yr in trial”, “Total Yield”, “%RB”, “Y1s”, “%RB 1s”, “%1s”, “Tuber/Plant”, “Length/Width”, “Specific Gravity”, “Hollow Heart”, “Brown Center”, “Vascular”, “Skin Color”, “Tuber Shape”, “Shape Uniformity”, “Scab”, “Ctrl Ave”, “GDD 1–60”, “GDD 61–90”, “GDD 91–end”
|

##  4 Discussion

We have applied a large collection of machine learning classification
techniques to manually collected Russet potato trial datasets. The main goal
was to investigate whether and how machine learning methods might provide
domain experts with faster and more efficient ways to identify potato
varieties for advancement to later trials. We discovered that although missing
data can pose a challenge, we can use imputation or strategic dropping of
features, as well as engineer new features, to obtain significant improvements
in the model classification performances. As illustrated in the results, the
large number of negative examples in our dataset resulted in strong
performance for predicting when to drop varieties from the trials. Although
this bias in the predictions can lead to false negatives, it also means that
varieties that will potentially fail in future trials are dropped earlier on,
making the process more efficient and economical. Further, our results suggest
that integrating machine learning techniques with the art of selection can
result in increased selection efficiency. As the variety trials are costly and
keeping a clone that is not worthy for one additional year can be cost
detrimental. Our results indicate, with availability of extensive data, an
integrated art of selection along with data-informed confirmation from machine
learning can be effective.

In conclusion, this work has demonstrated the potential of machine learning
methods in revolutionizing potato variety selection during trials. The
integration of varied data sources, coupled with advanced data-processing
techniques, present a pathway toward a more efficient and informed selection
process.

##  Appendix A Preprocessed Dataset Characteristics

Table A: Attributes for which data was collected and averaged across 4
replicates of each unique clone.

Attribute Name |  Attribute Description
---|---
Yield Total |  in hundredweight/acre. Sum of all different sizes, yield 2’s and culls where each has been multiplied by 8.31. Dataset ranges from 0-1510 with mean of 618. There are 47 missing values.
Rank |  An integer from 1 to number of varieties for a specific location and year. It ranks from highest (1) to lowest total yield. Ranges from 1-44. There are 47 missing values.
% RB |  Total yield divided by Russet Burbank’s total yield multiplied by 100. Ranges from 0-274. There are 47 missing values.
Yield No. 1’s |  Sum of all yields (in cwt) from which the potatoes are between 4 and 20 ounces. Ranges from 0-983. There are 47 missing values.
Rank (No. 1’s) |  An integer from 1 to number of varieties for a specific location and year. It ranks from highest (1) to lowest yield No 1’s. Ranges from 1-44. There are 47 missing values.
% RB (No. 1’s) |  Yield No 1’s divided by Russet Burbank’s yield No 1’s multiplied by 100. Ranges from 0-336. There are 47 missing values.
% No. 1’s |  Yield No 1’s divided by Yield Total multiplied by 100. Ranges from 0-93. There are 51 missing values.
$>$ 10 oz. |  Yield in cwt of potatoes above 10 oz. Sums of 10-20 oz (sometimes not provided) and over 20 oz. Ranges from 0-787. There are 143 missing values.
6-10 oz. |  Yield in cwt of potatoes between 6 and 10 oz. Ranges from 0-716. There are 144 missing values.
4-6 oz. |  Yield in cwt of potatoes between 4 and 6 oz. Ranges from 0-240. There are 143 missing values.
Yield No. 2’s + $>$ 20oz. |  Yield total minus Yield No. 1’s minus Yield Under 4 oz. minus Yield culls. Ranges from 0-429. There are 51 missing values.
Yield Under 4 oz. |  Yield in cwt of potatoes below 4 oz. Ranges from 0-807. There are 52 missing values.
Yield Culls |  Yield in cwt of discarded potatoes. Ranges from 0-178. There are 53 missing values.
Yield Over 20 oz. |  Yield in cwt of potatoes above 20 oz. There are 140 missing values.
Tuber/plant |  Total tuber count divided by the number of plants. There 276 missing values.
Average Tuber Size |  Measured in ounces. Ranges from 0-20. There are 175 missing values.
Length/Width Ratio |  Potato’s length divided by its width (computed from raw length and width, which are not separately included among the averages). Ranges from 0.52-3.5. There are 19 missing values.
Specific Gravity |  Potato’s air weight divided by the difference of its air and water weights (computed from raw air and water weights, which are not separately included among the averages). Ranges from 0-1.1. There are 18 missing values.
Fry Color Stem |  Measured in photovolts. Ranges from 14.4-72.9. There are 410 missing values.
Fry Color Bud |  Measured in photovolts. Ranges from 16.9-76.3. There are 410 missing values.
Hollow Heart |  Internal defect variable. Raw data is an expert-selected number from 1-10, multiplied by 10 to represent a percentage. Ranges from 0-90. There are 40 missing values.
Brown Center |  Internal defect variable. Raw data is an expert-selected number from 1-10, multiplied by 10 to represent a percentage. Ranges from 0-30. There are 40 missing values.
Black Spot Bruise |  Internal defect variable. Raw data is an expert-selected number from 1-10, multiplied by 10 to represent a percentage. Ranges from 0-87.5. There are 169 missing values.
Internal Brown Spot |  Internal defect variable. Raw data is an expert-selected number from 1-10, multiplied by 10 to represent a percentage. Ranges from 0-100. There are 40 missing values.
Vascular Discoloration |  Internal defect variable. Raw data is an expert-selected number from 1-10, multiplied by 10 to represent a percentage. Ranges from 0-100. There are 40 missing values.
Flower Color |  Plant characteristic. Ranges from 1-4 (white to pink). Variable scale. Should range from 1-5. There are 638 missing values.
Vine Size |  Plant characteristic. Ranges from 1.3-5.125. Variable scale. Should range from 1-5. There are 583 missing values.
Maturity |  Plant characteristic. Ranges from 0.1375-5.75. Variable scale. Should range from 1-5. There are 639 missing values.
Skin Color |  External characteristic. Ranges from 1.35-71.3. Variable scale. Should range from 1-5 (3 is yellow, 4 is brown). There are 290 missing values.
Russeting |  External characteristic. Ranges from 1-14.625. Variable scale. Should range from 1-5. There are 153 missing values.
Tuber Shape |  External characteristic. Ranges from 1-5 from spherical to long. There are 261 missing values.
Shape Uniformity |  External characteristic. Ranges from 1-5. There are 153 missing values.
Eye Depth |  External characteristic. Ranges from 1-5.5. Variable scale. Should range from 1-5. There are 119 missing values.
Greening |  External characteristic. Ranges from 1.625-5. Variable scale. Should range from 1-5. There are 153 missing values.
Growth Cracks |  External characteristic. Ranges from 1-5. There are 153 missing values.
Scab |  External characteristic. Ranges from 2.625-5. Variable scale. Should range from 1-5. There are 325 missing values.
Shatter Bruise |  External characteristic. Ranges from 2.16-20.83. Variable scale. Should range from 1-5. There are 176 missing values.
Bruise Data |  A string of number1 * number2 where number1 is the the diameter of the bruise and number2 is the depth of the bruise. This attribute is not included in the analysis as there is very little data for it.
|

## References

  *   * Araji and Love [2002] Araji, A.A., Love, S.: The economic impact of investment in the Pacific Northwest potato variety development program. American Journal of Potato Research 79, 411–420 (2002)
  * Akbarzadeh et al. [2018] Akbarzadeh, S., Paap, A., Ahderom, S., Apopei, B., Alameh, K.: Plant discrimination by support vector machine classifier based on spectral reflectance. Computers and electronics in agriculture 148, 250–258 (2018)
  * Breiman [2001] Breiman, L.: Random forests. Machine learning 45, 5–32 (2001)
  * Colantonio et al. [2022] Colantonio, V., Ferrão, L.F.V., Tieman, D.M., Bliznyuk, N., Sims, C., Klee, H.J., Munoz, P., Resende, M.F.R.: Metabolomic selection for enhanced fruit flavor. Proceedings of the National Academy of Sciences 119(7), 2115865119 (2022) https://doi.org/10.1073/pnas.2115865119
  * Chicco [2017] Chicco, D.: Ten quick tips for machine learning in Computational Biology. BioData mining 10(1), 35 (2017)
  * Camargo and Smith [2009] Camargo, A., Smith, J.: An image-processing based algorithm to automatically identify plant disease visual symptoms. Biosystems engineering 102(1), 9–21 (2009)
  * Dwivedi et al. [2021] Dwivedi, Y.K., Hughes, L., Ismagilova, E., Aarts, G., Coombs, C., Crick, T., Duan, Y., Dwivedi, R., Edwards, J., Eirug, A., Galanos, V., Ilavarasan, P.V., Janssen, M., Jones, P., Kar, A.K., Kizgin, H., Kronemann, B., Lal, B., Lucini, B., Medaglia, R., Le Meunier-FitzHugh, K., Le Meunier-FitzHugh, L.C., Misra, S., Mogaji, E., Sharma, S.K., Singh, J.B., Raghavan, V., Raman, R., Rana, N.P., Samothrakis, S., Spencer, J., Tamilmani, K., Tubadji, A., Walton, P., Williams, M.D.: Artificial Intelligence (AI): Multidisciplinary perspectives on emerging challenges, opportunities, and agenda for research, practice and policy. International Journal of Information Management 57, 101994 (2021) https://doi.org/10.1016/j.ijinfomgt.2019.08.002
  * Hinton [1990] Hinton, G.I.: Connectionist learning procedures, pp. 555–610. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA (1990)
  * Hastie et al. [2009] Hastie, T., Tibshirani, R., Friedman, J.H.: The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd edn. Springer, New York (2009)
  * Ke et al. [2017] Ke, G., Meng, Q., Finley, T., Wang, T., Chen, W., Ma, W., Ye, Q., Liu, T.-Y.: LightGBM: A highly efficient gradient boosting decision tree. NIPS’17, vol. 30, pp. 3149–3157 (2017)
  * Kurek et al. [2023] Kurek, J., Niedbała, G., Wojciechowski, T., Świderski, B., Antoniuk, I., Piekutowska, M., Kruk, M., Bobran, K.: Prediction of potato (solanum tuberosum l.) yield based on machine learning methods. Agriculture 13(12), 2259 (2023)
  * McMaster and Wilhelm [1997] McMaster, G.S., Wilhelm, W.: Growing degree-days: one equation, two interpretations. Agricultural and forest meteorology 87(4), 291–300 (1997)
  * [13] Ozguven, M.M., Yilmaz, G., Adem, K., Kozkurt, C.: Use of Support Vector Machines and Artificial Neural Network Methods in Variety Improvement Studies: Potato Example. Current Investigations In Agriculture and Current Research 6, 770–776
  * Pedregosa et al. [2011] Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., Duchesnay, E.: Scikit-learn: Machine learning in Python. Journal of Machine Learning Research 12, 2825–2830 (2011)
  * Rasmussen et al. [2006] Rasmussen, C.E., Williams, C.K., et al.: Gaussian Processes for Machine Learning vol. 1. Springer, New York (2006)
  * Schölkopf and Smola [2001] Schölkopf, B., Smola, A.J.: Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond vol. 4. MIT, Cambridge, MA (2001)
  * Steinbach and Tan [2009] Steinbach, M., Tan, P.-N.: knn: k-nearest neighbors. The top ten algorithms in data mining, 151–162 (2009)
  * Selvaraj et al. [2020] Selvaraj, M.G., Valderrama, M., Guzman, D., Valencia, M., Ruiz, H., Acharjee, A.: Machine learning for high-throughput field phenotyping and image processing provides insight into the association of above and below-ground traits in cassava (Manihot esculenta Crantz). Plant Methods 16(87), 19 (2020) https://doi.org/10.1186/s13007-020-00625-1
  * Tang et al. [2014] Tang, J., Alelyani, S., Liu, H.: Feature Selection for Classification: A Review. In: Data Classification: Algorithms and Applications vol. 56, pp. 37–64. CRC press, Boca Raton, FL (2014)
  * Van Buuren and Groothuis-Oudshoorn [2011] Van Buuren, S., Groothuis-Oudshoorn, K.: mice: Multivariate imputation by chained equations in r. Journal of statistical software 45, 1–67 (2011)
  * Williams and Rasmussen [2006] Williams, C.K., Rasmussen, C.E.: Gaussian Processes for Machine Learning vol. 2. MIT press, Cambridge, MA (2006)
  * Zhang [2004] Zhang, H.: The optimality of naive bayes. Aa 1(2), 3 (2004)