# Design and Development of an Autonomous Surface Vehicle for Water Quality
Monitoring

MM Rashid
Mechatronics Engineering
International Islamic University Malaysia
Kula Lumpur, Malaysia
<EMAIL_ADDRESS>
&Rupal Roy
Mechatronics Engineering
International Islamic University Malaysia
Kula Lumpur, Malaysia
<EMAIL_ADDRESS>
&Md Manjurul Ahsan
Industrial and Systems Engineering
University of Oklahoma
Norman, Oklahoma-73071
<EMAIL_ADDRESS>
&Zahed Siddique
Department of Aerospace and Mechanical Engineering
University of Oklahoma
Norman, Oklahoma-73071
<EMAIL_ADDRESS>
###### Abstract

Manually monitoring water quality is very exhausting and requires several
hours of sampling and laboratory testing for a particular body of water. This
article presents a solution to test water properties like electrical
conductivity and pH with a remote-controlled floating vehicle that minimizes
time intervals. An autonomous surface vehicle (ASV) has been designed
mathematically and operated via MATLAB & Simulink simulation where the
Proportional integral derivative (PID) controller has been considered. A PVC
model with Small waterplane area twin-hull (SWATH) technology is used to
develop this vehicle. Manually collected data is compared to online sensors,
suggesting a better solution for determining water properties such as
dissolved oxygen (DO), biochemical oxygen demand (BOD), temperature,
conductivity, total alkalinity, and bacteria. Preliminary computational
results show the promising result, as Sungai Pasu rivers tested water falls in
the safe range of pH ( 6.8-7.14) using the developed ASV.

_K_ eywords ASV $\cdot$ SWATH technology $\cdot$ Nomoto Model $\cdot$ WQI
$\cdot$ Autonomous Vehicle

## Abbreviations

ASV Autonomous Surface Vehicle
PID Proportional Integral Derivative
SWATH Small Waterplane Area Twin-Hull
WQI Water Quality Index

##  1 Introduction

The unmanned surface vessels (USVs) or the autonomous surface vehicles (ASVs)
are crafts that are designed to allow autonomous control of marine platforms
which are equipped with state-of-the-art sensors for carrying out a variety of
missions like marine environment monitoring, hydrologic survey, target object
searching, scientific study and so on [1]. These vehicles are developed for
shallow waters where it is difficult for survey vessels to reach and for areas
dangerous to human operators like quarry lakes, contaminated waters [2]. Till
date, many ASVs have been developed at academic labs, corporations, and
governments for oceanic survey and defense purposes [3]. Even though a large
number of proven prototypes of ASVs have been designed and tested, there are
only a few of them available in the industries for carrying out specified
tasks. Especially when compared to the autonomous underwater vehicles, the ASV
development area requires immense attention and further advancement [4].
The development of the first autonomous surface vehicles (ASV) dates back to
World War II and used primarily for military purposes [5]. In the year 1946,
ASVs were widely implemented in postwar applications, particularly to collect
radioactive samples of water after the two atomic tests—Able and Baker— in the
Pacific Ocean. In the 1950s-era, US Navy Mine Defense Laboratory’s project
DRONE constructed and tested a remotely operated minesweeping boat. By the
1960s remotely operated boats were used for missile firing purposes. The naval
use of the unmanned surface vehicles has evolved over the years and today, the
navy operates several USVs as target drones, including the Mobile Ship Target
(MST), the QST-33, and QST-35/35A SEPTAR Targets, and the (HSMST) High-Speed
Maneuverable Seaborne Target [6].
One of the first Autonomous surface vehicles (ASVs) named ARTEMIS was
developed in 1993 at the MIT Sea Grant Program [7]. Designed as a replica of a
fishing trawler, its main use was testing navigation and control systems of
the ASVs and was equipped with an electric motor and a servo actuated ruder.
Here Proportional-plus derivative (PD) control system (Fig. 1) was used for
simple heading control. Although ARTEMIS worked well as a test platform, it is
not suitable for research in the coastal zone and the open ocean due to its
small size and heading control. However, though ARTEMIS had limitations it
gives a better concept about ASC [8].

Figure 1: Block diagram of control system

A very important achievement in the field of autonomous surface vehicle
development was the designing of the four surface crafts known as SCOUT
(Surface Craft for Oceanographic and Undersea Testing) built by the MIT
Department of Oceanic Engineering. It was a single hull kayak system design
which makes it simple, robust, versatile, and had improved operational
utility. The main concern of the SCOUT developers was low cost and high
flexibility. Striking features about the SCOUT were that it could be
controlled autonomously by preprogramming its controller; moreover, it could
be controlled by an external computer via Wi-Fi or even via a computer that
could give instructions directly to the propulsion and steering systems. In
addition, Kayak design is popular due to its convenient mounting and loading
[9]. There are catamarans (twin hulls) [10] and trimarans (triple hulls) [11]
system ASV, which are often chosen because of their higher system stability,
payload capacity, redundancy, and fewer risks of overturning in coastal water
[12]
Autonomous underwater vehicles (AUV) are compact, battery-powered submersibles
equipped with their computers and capable of performing a wide range of tasks
autonomously underwater [13]. AUVs have only been used in a semi-autonomous
mode until recently, where they are constantly tracked and monitored [14].
There are ongoing research efforts to provide real-time localization of
multiple AUVs by using ASVs. The ASVs equipped with underwater acoustic
modems, GPS, Wi-Fi, and 802.11b wireless Ethernet which used to achieve moving
baseline data and help in testing adaptive behavior algorithms. The ASVs have
successfully helped in acquiring position estimation and have increased the
effectiveness of the AUC experiments [9]. The DELFIM ASC was developed to
automatically record marine data and as a communication relay for a submarine
and an auxiliary ship. It facilitates the transfer of sonar and acoustic data
via a vertically designed acoustic communication route.(Pascoal et al., 2000).
The INFANTE is a self-contained AUV capable of operating at a maximum depth of
500 meters. The vehicle is a significant redesign of the MARIUS AUV, which was
developed under the EU’s MAST-I and MAST-I1 programs. Through the integration
of motion sensors, the INFANTE AUV features an innovative navigation system
[15].
ASIMOV project is an advanced technological system where DELFIM ASC and
INFANTE AUV two robotic ocean vehicles, were used designed, and built by the
Institute for Systems and Robotics of the IST. It was developed for an
acoustic communication link with the AUVs and with other underwater vessels
using the ASVs. To serve this purpose a 3.5m catamaran with an indigenous
wing-shaped central structure was developed. It was equipped with acoustic
transducers, two-bladed propellers powered by electrical motors, and Doppler
velocimeter. Normal ASV’s are low in speed but catamaran-style vehicles are
more stable and faster for their monohull designs [8].
Water is a natural resource that must be preserved, protected, and monitored
continuously as it is the basic elements, necessary for the survival of not
only humans but all kinds of life forms on earth [16]. Water quality
monitoring is highly essential for tracking pollutants, the presence of
harmful elements, and in turn, ensuring clean non-toxic water supply for
humans as well as aquatic life [17]. The various parameters that used to
monitor the water quality includes Dissolved Oxygen (DO), Biochemical Oxygen
Demand (BOD5), Chemical Oxygen Demand (COD), Suspended Solids (SS), Ammoniacal
Nitrogen (AN), pH and so on. WQI (Water Quality Index) is used to classify the
monitored river water quality. The WQI is based on the Interim National Water
Quality Index (INWQS), which is defined as a set of standards based on
beneficial uses of water. To classify the river water quality that is
monitored there is a Water Quality Index. A Water Quality Index (WQI) assigns
quality value to a set of rough data that is collected from water samples. The
main purpose of WQI is to interpret the collected data and make it simple for
people to understand, whether the water quality is good or bad [18].
HydroNet ASV is a small-sized ASV, designed for monitoring the coastal water
quality which detects heavy metal concentrations such as chrome: Cr (VI),
mercury: Hg (II), and cadmium: Cd (II) from water in real-time using custom-
made miniaturized onboard sensors. The USV is designed for long-range
missions, lodging an onboard water analysis system. It was tested in the
Livorno coastal area distance of 12 682.6 m [19, 20]. Subsequently, the
HydroNet modification was used to collect water samples up to 50 m from the
water column. The system was retested in field trials at Leverkusen, but they
do not perform any chemical analyzes of the water, which was left for future
work [21]. Springer vehicle is another USV designed as a double-hole vessel
with a medium seaplane and can be used in a variety of ways as a mission water
profile and payload. It was developed for environmental monitoring and
pollutant tracking. An integrator sensor was used for navigation purposes [10]
At present low-resolution, water quality monitoring is being carried out by
various laboratories [22]. However, there are many disadvantages of the manual
approach [23]. The collection of data by this method is not continuous in
space and time and thus, occasionally occurring pollutants are missed. Also,
this method is time-consuming and expensive as it involves frequent visits by
an individual to specific. Specific water properties such as dissolved oxygen
content and oxidation-reduction potential are susceptible to change in
pressure and altitude, and therefore these parameters must be measured on-spot
for better accuracy. Laboratory testing approach provides slow, inaccurate and
sparse results which are difficult to be interpreted and tabulated [17]
The autonomous surface crafts are becoming exceedingly popular for water
quality monitoring especially in coastal and estuarine waters because of their
ability to provide real-time on- spot data without the need of taking the
water samples to the laboratory for testing and data generation [24]. These
unmanned vehicles equipped with in-situ instrumentation are capable of
collecting data from places inaccessible to the human population like water
with high algal concentration. These unmatched advantages of collecting spot
data with real-time transmission and reaching inaccessible areas have led to
the increased popularity of ASVs in the field of water quality surveillance
[25]. Every day along with the coastal areas, harsh winds, swell waves, and
other phenomena control sand transport and change ocean morphology thus
playing a vital role in the dynamic behavior of coastal regions. This
constantly changing ocean morphology needs continuous monitoring which is
achieved through Bathymetric surveys that generate maps for the surveyed
areas. These maps illustrate ocean bed topography and help in creating vessel
navigation charts [26]. The ever-growing demand for bathymetric surveys and
the inaccessibility of some coastal areas to the human population have created
the demand for autonomous surface crafts [27]. Various autonomous surface
crafts like the solar-powered OASIS platform that investigated the nature and
extent of harmful algal blooms, the Wave Glider powered by wave hydel energy
and the ASV developed at ETH Zurich with a winch mechanism that measures
varying oceanic depths (up to 130m deep) [28].
From the above discussion, we can see that many of these surface crafts
discussed are exceptionally high cost and developed on a very large scale thus
restricting their usage to large scale industrial projects. Also, many of the
surface crafts are semi-autonomous and have short battery life [29]. Thus,
there is a need for the development of an autonomous surface craft that can
operate in riverine and estuarine environments and provide cost-effective and
portable long-duration data gathering. Wireless sensor networks (WSNs) were
developing swiftly withinside the beyond few years [30]. Lots of studies has
been carried out on WSN communications, WSN strength conservation, WSN routing
algorithms, etc. However, maximum studies particularly specialize in
terrestrial sensor networks. Research on underwater sensor networks is
constrained. Considering that constrained into account, in this study, water
first-rate tracking systems have been accessed that can assist with water
pollutants detection and discharge of poisonous chemical substances and
infection in water. As a consequence, temperature, pH, dissolved oxygen,
connectivity and turbidity parameters are monitored in river/lake water first-
rate tracking systems.
The summary of the technical contribution can be express as follows:

  1. 1.

Design a suitable mechanical structure for an autonomous surface vehicle,
capable of floating and maneuvering on the surface of a water body;

  2. 2.

Develop a sensory mechanism for the ASV to measure water quality parameters on
the spot;

  3. 3.

Develop navigation and control techniques for the surface vehicle to aid in
monitoring water quality and collecting bathymetric data and;

  4. 4.

Evaluate the performance of the autonomous surface vehicle

The remainder of the section is organized as follows: Section 2 discussed
materials and methods. Section 3 describes our study’s findings and examines
the general conclusions. Section 4 concludes by summarizing the general
findings.

##  2 Materials and methods

###  2.1 Mechanical Design of ASV

The Small Waterplane Twin Hull Design (SWATH) design is chosen for this
autonomous surface vessel (Fig. 2). The design is selected to reduce roll and
pitch disturbances which lead to errors in on-spot measurement. SWATH design
gives better wave disturbance rejection and excellent stability during roll
and pitch motions as it has less hull volume at the waterline, giving it
lesser buoyancy change during wave interaction.

Figure 2: SWATH design gives stability to the vehicle

The following Table 1 and 2 shows the data collected from the tests:

Table 1: Dynamic analysis data of the SWATH style ship ‘wasp’ [31]) | Freeboard | Cycles | Displacement | Time | Period
---|---|---|---|---|---
| in | n | B1 | B(n+1) |  $\Delta$ t(s) | P
Roll | 16 | 6 | 7 | 3.5 | 37.90 | 6.32
12 | 5 | 10.5 | 4.5 | 36.33 | 7.27
8 | 6 | 13 | 4.5 | 41.60 | 6.93
Pitch (Black) | 16 | 2 | 1.5 | 0.5 | 12.61 | 6.31
12 | 4 | 3.5 | 0.75 | 27.66 | 6.91
8 | 4 | 3.25 | 0.75 | 25.74 | 6.44
Pitch (Front) | 16 | 2 | 1.5 | 0.5 | 12.26 | 6.13
12 | 5 | 2.5 | 1 | 34.46 | 6.89
8 | 6 | 2.75 | 1 | 38.48 | 6.41
Table 2: Dynamic analysis data of the SWATH style ship ‘wasp’ [31] Logarithmic decrement |  Damping ratio |  Damped frequency |  Natural frequency |  Stiffness |  Damping |  Critical damping constant
---|---|---|---|---|---|---
$\delta$ (delta) |  $\zeta$ (zeta) |  $\omega$ d |  $\omega$ n |  k (Ibs/ft) |  c(lbs-sec/ft) |  cc
0.116 |  0.00919 |  0.99459 |  0.99464 |  24.58 |  0.454 |  49.42
0.169 |  0.01348 |  0.86474 |  0.86482 |  19.74 |  0.616 |  45.66
0.177 |  0.01407 |  0.90619 |  0\. .90627 |  22.96 |  0.713 |  50.66
0.549 |  0.04367 |  0.99638 |  0.99733 |  24.71 |  2.164 |  49.56
0.385 |  0.03063 |  0.90876 |  0.90919 |  21.82 |  1.470 |  48.00
0.367 |  0.02916 |  0.97641 |  0.97682 |  26.67 |  1.592 |  54.61
0.549 |  0.04367 |  1.02482 |  1.02580 |  26.14 |  2.226 |  50.97
0.183 |  0.01458 |  0.91166 |  0.91166 |  21.94 |  0.702 |  48.14
0.169 |  0.01342 |  0.97960 |  0.97969 |  26.83 |  0.735 |  54.77

To find the stability of the boat, the following linear second-order equation
was used:

$m\ddot{x}+c\dot{x}+kx=f(t)$ (1)

Where, ‘m’ is the mass of the system, ‘c’ is the damping, which is a factor of
the horizontal surface area and slows vertical motion, and ‘k’ is the
stiffness which is a factor of the change in displacement versus the change in
height. For a typical system, high values would imply a significant resistance
to input forces; however, with aquatic vessels, higher damping and, more
specifically, high stiffness correspond to greater wave tracking. For the
SWATH case, both values should be low compared to the mass, which they are.
Thus, it can be predicted that the vessel will have a low response to waves
for the targeted sea state.
By testing at different heights, the optimal loading and operating range can
be determined. For each of the test settings, the middle data set (which
corresponds to 12”free board) gives the lowest stiffness. This is because it
is using the thinner section of the struts, whereas the lower setting ended up
using the reserve flotation, and the higher setting brings the pontoons out of
the water in extreme cases. Using the equation 2, the system is defined as
stable if both roots have negative and real parts, which occurs when m, c, and
k have the same sign and is a rare possibility in the case of catamaran and
kayak style boats.

$x=\frac{-b\pm\sqrt{b^{2}-4ac}}{2a}=s1,\ s2$ (2)

For the SWATH scenario, all three are positive, indicating that the SWATH
system is more stable here in this range. After making the design choice, the
surface vehicle ‘s main hull is made from various shapes and types of PVC
pipes which are positioned together at 90° angles to form a structure, capable
of staying afloat on water and carrying a payload of up to 5kgs. On top of the
PVC, boat structure is mounted as a platform to house the electrical
components that are the microcontroller and the sensors.
The following components and materials are chosen to conduct the experiment:

  * •

The Hull: Hull is composed of Polyvinyl Chloride (PVC) pipes of various
diameters. PVC pipes are ideal for building the boat structure as they are
lightweight and can be easily installed. They can sustain low temperatures and
are greatly resistant to corrosion. Being low cost, biological, and chemical
resistant and having easy workability, it is being used in a wide variety of
applications.

  * •

Propulsion System: The propulsion system is composed of four Seabotix BTD
thrusters. The thrusters are brushless DC motors encased in a watertight
housing. These thrusters help in forwarding movement and also aid in turning
and maneuvering in narrow spaces (Fig. 3). Platform: A platform made of wood
is mounted on top of the pipe arrangement completing the structure of the
vehicle. It is used to house the microcontroller and other electric equipment
used. A sturdy wooden platform was chosen as the platform had to be cut in
several places for the installment of sensors and other connections.

  * •

Electronic Component Enclosure: Electronic components form an integral part of
the autonomous vehicle system and need to be given proper protection during
the operation. A watertight plastic case is used for the placement of all the
electrical equipment in the system. Two pipe casings adjacent to each thruster
are provided for the Electrical Conductivity and pH sensor for water quality
monitoring.

Figure 3: Block diagram of propulsion system

###  2.2 Navigation and Data Transmission system of ASV

The propelling force that drives the vehicle comes from the four thrusters.
The sensors (EC sensor, pH sensor, Depth Sensor) are the components that
provide the data to be transmitted. The microcontroller, the GPS receiver, and
the compass would be interfaced with the Arduino microcontroller which will
perform all data processing. Programming in the Arduino environment is much
easier as compared to the conventional C/C++ programming. Fig. 4 demonstrates
the electronics systems diagram used to develop the autonomous vehicle.

Figure 4: Electronics System Diagram of the Autonomous Surface Vehicle

Since the boat is an autonomous surface vehicle, there is no tether between
the boat at the water body and the computer on the shore. The Arduino
microcontroller is preprogrammed the location coordinates from where the water
samples need to be tested and it converts numerical values to pulse width
modulation (PWM) signals for the motor controller and the boat moves. The
sensors interfaced with the Arduino at the water body provide data to it and
through the Xbee modules communicating wirelessly with each other, the data is
automatically transferred from the water body onto the shore. Thus,
transmission between the Xbee Shield and the Xbee connected to the computer
onshore helps to transfer sensor readings in real-time.

###  2.3 Kinematic Analysis of the ASV

The first step in modeling the vehicle is to choose the coordinate reference
frames. Two coordinate frames are chosen for the autonomous vehicle. One is
the fixed frame of reference (XYZ) such that at time t = 0, the CG (center of
gravity) of the vehicle will be at the origin. The second frame chosen is the
body-fixed frame (BFF) such that its origin lies at the CG of the vehicle as
shown in the Fig. 5 below:

Figure 5: The body-fixed axis and the Global axis of the ASV

The second step is to assign the degrees of freedom to our vehicle. Following
the popular assumption of neglecting the heave, pitch, and roll motions which
does not have any effect on the accuracy of the vehicle while maneuvering to
reach its target. Thus, the configuration vector of the fixed frame XYZ for
the body frame BFF is given as:

$\eta(t)=[X,Y,\uppsi],t\geq 0$ (3)

The surge (movement along $X_{b}$ axis) designated by u(t), sway (movement
along $Y_{b}$ axis) designated by v(t), and yaw (rotation about the $Z_{b}$
axis) designated by r(t) are the three motions that are taken into
consideration. The velocity vector is then represented as:

$V(t)=[u,v,r],t\geq 0$ (4)

Taking angle of trim ($\theta$) and angle of the roll ($\upgamma$) as
negligible, the rotation matrix from the fixed to the body reference frame is
given as:

$J\left(\eta\right)=\left[\begin{matrix}cos\psi&-sin\psi&0\\\
sin\psi&cos\psi&0\\\ 0&0&1\\\ \end{matrix}\right]$ (5)

From equations (3), (4) & (5):

$\frac{d(\eta\left(t\right))}{dt}=\dot{\eta}\left(t\right)=J\left(\eta\right)V\left(t\right)$
(6)

Equation (6) depicts the relation between $\eta$ (t) and V(t) and these
together represent the kinematic equations of the ASV.

###  2.4 Motion Analysis of the ASV

The next step is to generate the ASV’s equations of motion, which is
accomplished by taking into account the external forces and torques acting on
water. Taking the forces and moments along the three directions of the fixed
body frame, equations can be developed as follows: For Surge (movement along
the $X_{b}$ axis),

$X_{b}=m(\dot{u}-vr-yg\dot{r}-x_{g}r^{2})$ (7)

For Sway (movement along the $Y_{b}$ axis),

$Y_{b}=m(\dot{v}-ur-xg\dot{r}-ygr^{2})$ (8)

For Yaw (rotation about the $Z_{b}$ axis),

$N=Iz\dot{r}+m[xg(\dot{r}+ur)-yg(\dot{u}-vr)]$ (9)

$X_{b}$, $Y_{b}$, N are the forces and the moment acting on the vehicle and
$x_{g}$ and $y_{g}$ are the distances from the origin of the fixed body
reference to the CG of the vehicle. When $x_{g}$ and $y_{g}$ $\rightarrow 0$,
the simplified matrix form of the motion equations can be written as:

$M\dot{V}(t)+[C(v)+D(v)]V(t)+g[\eta(t)]=T_{R}(t);t\geq 0$ (10)

Where M represents the mass- inertia matrix. $C[v(t)]\epsilon R^{3\times 3}$
represents the coriolis and centripetal matrix. $D[v(t)]\epsilon R^{3}$
represents the damping vector, $g[\eta(t)]\epsilon R^{3}$ is the gravitational
force and moments vector. Finally $T_{R}(t)=[X_{b}(t),Y_{b}(t),N(t)]^{T}$ is
the input vector of the forces and moments. The mass- inertia and Coriolis
matrix involve the rigid body mass and added mass which occur due to the force
of the fluid in contact with the vehicle. Thus, matric M and C [v(t)] can be
re-written as:

$M=M_{RB}+M_{AM}$ (11)

where,

$M_{RB}=\left[\begin{matrix}m&0&0\\\ 0&m&0\\\ 0&0&Iz\\\ \end{matrix}\right]$

$C_{RB}=\left[\begin{matrix}0&-mr&0\\\ mr&0&0\\\ 0&0&0\\\ \end{matrix}\right]$

Since we are assuming the heave motion to be negligible in this case, the
gravitational matrix $g[\eta(t)]=0$.

###  2.5 Dynamic Analysis of the ASV

To develop a complete mathematical model for the ASV, it is important to
consider the hydrodynamic forces that are the drag and the various kinds of
frictional forces acting on the vehicle. First, the damping forces in the
$X_{b}$ direction acting on the hull can be given as:

$F_{h},X_{b}(u,r)=C_{f}\rho A_{h}U\mid U\mid$ (12)

This force on the hull is due to the viscous drag and depends on Cf, the
frictional resistance coefficient, $\rho$, the density of water in $m^{3}$,
wetted area of the hull, $A_{h}$ and the U, the distance from CG to the port
side of the hull in the $Y_{b}$ direction. Now, integrating the damping forces
for both hulls to get the total damping force in the $Y_{b}$ direction:

$F_{h},Y_{b}(v,r)=\int_{-dax}^{dfx}{1/2CDhTh\rho v(x)\mid v(x)\mid dx}$ (13)

Here,

$v\left(x\right)=v+\ \frac{vbow-vstern}{Lh}$

Having completed the damping force analysis, the next step is to analyze the
thrust force that acts on the vehicle. A propulsion system aids the movement
of the vehicle to the desired location at a particular velocity and
acceleration. It is the propellers that generate thrust. T that drives the
vehicle in water which is given by:

$T=K_{T}\rho d^{4}n^{2}$ (14)

The torque of the shaft attached to the propeller is represented by the
equation:

$Q=K_{Q}\rho d^{5}n^{2}$ (15)

The input forces and the moments put together to give us the thrust vector
$T_{R}$:

$T_{R}=[XYN]$ (16)

Let $T_{P}$ and $T_{S}$ be the thrusts delivered by the port side and star
side of the vehicle respectively and $\alpha_{p}$ and $\alpha_{S}$ be the
inclinations to the $X_{b}$ axis. Also, $d_{1}$ and $d_{2}$ are taken as
distanced in $X_{b}$ and $Y_{b}$ directions. Thus, the thrust vector is given
by:

$X=T_{P}cos\alpha_{p}+T_{s}cos\alpha_{S}$ (17)
$Y=T_{P}sin\alpha_{P}-T_{s}sin\alpha_{S}$ (18) $N=X.d_{1}+Y.d_{2}$ (19)

From eq (8) we have:

$\dot{V}(t)=-M^{-1}[C(v)+D(v)]V(t)+M^{-1}TR$ (20)

Defining state vector:

$X=[\eta^{T}V^{T}]^{T}$ (21)

From (6) and (18) State Space Representation of the ASV is given as:

$\dot{\mathbf{X}}=f(X)+BT_{R}$ (22) $B=\left[\begin{matrix}0\\\ M^{-1}\\\
\end{matrix}\right]$ (23) $f(X)=\begin{bmatrix}J(\eta)V(t)\\\
-M^{-}1[C(v)+D(v)]V(t)\\\ \end{bmatrix}$ (24)

###  2.6 Nomoto Model for Heading Control of the ASV

After the kinematic and dynamic analysis of the ASV, the next step is to
develop a suitable controller for its heading control. For this purpose, a
Nomoto Model is chosen which is a ship steering model utilizing first order
second-order models for ship autopilot design. The Nomoto Model is highly
popular in steering autonomous vehicles as it is simple and accurate, which is
highly essential considering the nonlinear and complex hydrodynamics involved
with boat steering.
The transfer function of the second order Nomoto Model is given by:

$\frac{r(s)}{\delta(s)}=\frac{Kr(1+T3)}{(1+T1s)(1+T2s)}$ (25)

where $K_{r}$ is the yaw rate gain, and $T_{1}$, $T_{2}$, and $T_{3}$ are time
constants. The numerical values for these constants are adapted. After
substituting the numerical value of the constants, the transfer function
obtained is given as:

$\frac{r(s)}{\delta(s)}=\frac{1.9103s+5.799}{0.3037s^{2}+0.7488s-1}$ (26)

Using the second-order Nomoto Model and plotting the root locus and step
response showed that the system was unstable as one of the poles fell on the
right-hand plane. To address this issue, a PID controller was utilized in
conjunction with the Nomoto Model. After plotting the root locus, step
response, and bode plots, it was observed that the system became stable due to
the adoption of the PID controller as shown in Fig. 6.

Figure 6: Root Locus of the stable $2^{nd}$ Order Nomoto Model

Therefore, the Nomoto Model is used for the heading control of the autonomous
surface vehicle in case the vehicle is thrown off track due to wind, heavy
rain, and other disturbances. The controller helps the vehicle to follow its
desired course irrespective of the physical disturbances by receiving feedback
from the compass.

##  3 Result and Discussions

The first step in determining the performance of the ASV was to perform its
motion analysis using the SolidWorks software. This motion analysis gives an
idea of the stability of the ASV on the surface of the water which is the most
important feature of a waterborne vehicle. The following graphs (Fig. 7)
depict the angular displacement (in terms of pitch and roll) of the ASV with
time. The slow decrease in a response implies low damping and stiffness which
is characteristic of a SWATH style boat as mentioned earlier. Fig. 7 shows
that within 50 seconds, the side-to-side response of the boat decreases, and
it gains stability.

Figure 7: Side-to-side response of the ASV with 8” of freeboard (distance from
the waterline to the boat top)

The separate responses of the bow (front side) and the stern (rear side) to
the waves are also obtained depicting individual stability as shown in Fig. 8
and 9 respectively.

Figure 8: Side-to-side response of the bow of the ASV with 8” of freeboard
Figure 9: Side-to-side response of the stern of the ASV with 8” of freeboard

After analyzing the movement of the boat in water, the next step is to collect
data. For this purpose, water samples were collected from the Sungai Pasu
river Malaysia and tested for pH and Electrical Conductivity at the
Environmental Laboratory at the Biotechnology Engineering Department. Fig. 10
displays the developed ASV that measure the quality parameters.

Figure 10: ASV measuring Water Quality Parameters

Several readings were taken at different times and recorded. After that, the
ASV was used to measure the pH and electrical conductivity of the river water.
The sensor data was wirelessly transmitted from the boat to the computer
simultaneously via email using the XCTU software. During the water monitoring,
the compass sends its readings continuously to the station computer at the
shore, giving the heading of the ASV. This reading is in degrees as measured
from the North. The compass is placed on the ASV such that the stern (front)
of the boat points to north, thus any deviation of the stern from its path is
recorded and transmitted by the compass.
After testing the ASV at the water, a comparison was made between the standard
pH value vs transmitted pH value (as shown in Fig. 11) and the standard EC
value vs transmitted EC value (as shown in Fig. 12) of the ASV. data collected
at the lab using standard method and values of pH and EC obtained and
transmitted by the ASV. Fig. 9 depicts that pH values obtained from the lab
are very close to the values obtained by the ASV. However, there was a
disparity of about 1-2 mS /cm (millisiemens per centimeter) between the
Electrical Conductivity (EC) data collected by the two methods. The main
reason for this was that data in the lab was collected at a low temperature of
about 21◦C and hence the EC of water was lower as compared to the values
collected by the ASV at an outside temperature of around 31◦C. Note that,
Electrical Conductivity is temperature-dependent (increasing approximately
2-3% per degree Celsius).

Figure 11: Comparative analysis of pH results Figure 12: Comparative analysis
of Conductivity results

The Table 3 below shows the Malaysian Water Quality Index, which assigns an
acceptable range to the various water quality parameters. The purpose of a WQI
is to summarize large amounts of water quality data for a specific river into
simple terms. This makes it easily understandable for communities in water
quality monitoring. The acceptable pH for various classes of water depending
on their use are in the range of 6.5 to 9 and thus, it can be said that Sungai
Pasu River water tested by the ASV falls in the safe range of pH ( 6.8-7.14).
However, the Electrical Conductivity of the river water (13-14mS/cm), when
compared to the Water Quality Index, is far above the acceptable range of 1-6
mS/cm. High Electrical Conductivity indicates high dissolved-solids
concentration; dissolved solids can affect the suitability of water for
domestic, industrial, and agricultural uses and can even be hazardous to
aquatic life. Additionally, high dissolved-solids concentration can cause
deterioration of plumbing fixtures and appliances [18].

Table 3: 3 Malaysian Water Quality Index [18] Parameters | Unit | Classes
---|---|---
Ammoniacal Nitrogen | mg/l | 0.1 | 0.3 | 0.3 | 0.9 | 2.7 | >2.7
BOD | mg/l | 1 | 3 | 3 | 6 | 12 | >12
COD | mg/l | 10 | 25 | 25 | 50 | 100 | >100
DO | mg/l | 7 | 5-7 | 5-7 | 3-5 | <3 | <1
pH | mg/l | 6.5-8.5 | 6.5-9.0 | 6.5-9.0 | 5-9 | 5-9 | -
Color | TUC | 15 | 150 | 150 | - | - |
Elec. Conductivity |  $\mu$ S/cm | 1000 | 1000 | - | - | 6000 | -

From the data collected by the ASV, followed by its analysis, it is clear that
the IIUM river needs water-treatment processes, such as coagulation,
flocculation, sedimentation, and disinfection, to remove excessive dissolved
solids from it and thus improve its quality.

##  4 Conclusions

In this work, an autonomous surface vehicle (ASV) was designed and tested to
carry out water quality monitoring. The kinematic and dynamic modeling of the
autonomous vehicle was done using the universal motion and moment equations.
Also, a heading controller was developed to keep the vehicle on its course
during harsh winds and other disturbances. The SWATH style structure of the
vehicle gave it more stability while carrying out missions on the water
surface. The use of Smart Drive motor drivers for driving the thrusters helped
each thruster to move independently thus making differential steering possible
for the vehicle. The main controller of the vehicle is the Arduino Mega 2560
which gave instructions to the thrusters to move in the desired direction. The
navigation system of the vehicle which consists of a GPS receiver and compass
was interfaced with the Arduino and wireless communication between the ASV and
control center was made possible by the use of Xbee modules. The developed ASV
can be used to collect field data for water quality assessment through
wireless remote sensing. In addition, it can be used for field measurements in
inaccessible and toxic waters. Another important contribution of ASV could be
the rapid measurement of water quality after natural disasters such as floods
and hurricanes. Future works include but are not limited to ASV models
integrated with machine learning techniques [32, 33] and artificial
intelligent-based ASD data analysis.

## References

  * [1] Fletcher Thompson and Damien Guihen.  Review of mission planning for autonomous marine vehicle fleets.  Journal of Field Robotics, 36(2):333–354, 2019.
  * [2] Jianhua Wang, Wei Gu, and Jianxin Zhu.  Design of an autonomous surface vehicle used for marine environment monitoring.  In 2009 International Conference on Advanced Computer Control, pages 405–409. IEEE, 2009.
  * [3] Anete Vagale, Rachid Oucheikh, Robin T Bye, Ottar L Osen, and Thor I Fossen.  Path planning and collision avoidance for autonomous surface vehicles i: a review.  Journal of Marine Science and Technology, pages 1–15, 2021.
  * [4] Justin E Manley.  Unmanned surface vehicles, 15 years of development.  In OCEANS 2008, pages 1–4. Ieee, 2008.
  * [5] SJ Corfield.  Unmanned surface vehicles and other things.  In Proceedings of the Unmanned Underwater Vehicle Showcase 2002 Conference, Southampton, UK, pages 83–91, 2002.
  * [6] Volker Bertram.  Unmanned surface vehicles-a survey.  Skibsteknisk Selskab, Copenhagen, Denmark, 1:1–14, 2008.
  * [7] MA Hinostroza, Haitong Xu, and C Guedes Soares.  Cooperative operation of autonomous surface vehicles for maintaining formation in complex marine environment.  Ocean Engineering, 183:132–154, 2019.
  * [8] Massimo Caccia.  Autonomous surface craft: prototypes and basic research issues.  In 2006 14th Mediterranean Conference on Control and Automation, pages 1–6. IEEE, 2006.
  * [9] Joseph Curcio, John Leonard, Jerome Vaganay, Andrew Patrikalakis, Alexander Bahr, David Battle, Henrik Schmidt, and Matthew Grund.  Experiments in moving baseline navigation using autonomous surface craft.  In Proceedings of OCEANS 2005 MTS/IEEE, pages 730–735. IEEE, 2005\.
  * [10] W Naeem, T Xu, R Sutton, and A Tiano.  The design of a navigation, guidance, and control system for an unmanned surface vehicle for environmental monitoring.  Proceedings of the Institution of Mechanical Engineers, Part M: Journal of Engineering for the Maritime Environment, 222(2):67–79, 2008.
  * [11] Yan Peng, Jian-da Han, and Qing-jiu Huang.  Adaptive ukf based tracking control for unmanned trimaran vehicles.  International Journal of Innovative Computing, Information and Control, 5(10):3505–3516, 2009.
  * [12] Sable Campbell, Wasif Naeem, and George W Irwin.  A review on improving the autonomy of unmanned surface vehicles through intelligent collision avoidance manoeuvres.  Annual Reviews in Control, 36(2):267–283, 2012.
  * [13] Roy A Armstrong, Oscar Pizarro, and Christopher Roman.  Underwater robotic technology for imaging mesophotic coral ecosystems.  In Mesophotic Coral Ecosystems, pages 973–988. Springer, 2019.
  * [14] Elgar Desa, R Madhan, and P Maurya.  Potential of autonomous underwater vehicles as new generation ocean data platforms.  Current science, pages 1202–1209, 2006.
  * [15] Carlos Silvestre and A Pascoal.  Control of the infante auv using gain scheduled static output feedback.  Control Engineering Practice, 12(12):1501–1509, 2004.
  * [16] A Inyinbor Adejumoke, O Adebesin Babatunde, P Oluyori Abimbola, A Adelani Akande Tabitha, O Dada Adewumi, A Oreofe Toyin, et al.  Water pollution: effects, prevention, and climatic impact.  Water Challenges of an Urbanizing World, 33:33–47, 2018.
  * [17] Aravinda S Rao, Stephen Marshall, Jayavardhana Gubbi, Marimuthu Palaniswami, Richard Sinnott, and Vincent Pettigrovet.  Design of low-cost autonomous water quality monitoring system.  In 2013 International Conference on Advances in Computing, Communications and Informatics (ICACCI), pages 14–19. IEEE, 2013.
  * [18] Zaki Zainudin.  Benchmarking river water quality in malaysia.  Jurutera, 12:15, 2010.
  * [19] Gabriele Ferri, Alessandro Manzi, Francesco Fornai, Barbara Mazzolai, Cecilia Laschi, Francesco Ciuchi, and Paolo Dario.  Design, fabrication and first sea trials of a small-sized autonomous catamaran for heavy metals monitoring in coastal waters.  In 2011 IEEE International Conference on Robotics and Automation, pages 2406–2411. IEEE, 2011.
  * [20] Gabriele Ferri, Alessandro Manzi, Francesco Fornai, Francesco Ciuchi, and Cecilia Laschi.  The hydronet asv, a small-sized autonomous catamaran for real-time monitoring of water quality: From design to missions at sea.  IEEE Journal of Oceanic Engineering, 40(3):710–726, 2014.
  * [21] Francesco Fornai, Gabriele Ferri, Alessandro Manzi, Francesco Ciuchi, Francesco Bartaloni, and Cecilia Laschi.  An autonomous water monitoring and sampling system for small-sized asvs.  IEEE Journal of Oceanic Engineering, 42(1):5–12, 2016.
  * [22] Hedy M Aardema, Machteld Rijkeboer, Alain Lefebvre, Arnold Veen, and Jacco C Kromkamp.  High-resolution underway measurements of phytoplankton photosynthesis and abundance as an innovative addition to water quality monitoring programs.  Ocean Science, 15(5):1267–1285, 2019.
  * [23] Yiheng Chen and Dawei Han.  Water quality monitoring in smart city: A pilot project.  Automation in Construction, 89:307–316, 2018.
  * [24] Naomi A Ubina and Shyi-Chyi Cheng.  A review of unmanned system technologies with its application to aquaculture farm monitoring and management.  Drones, 6(1):12, 2022.
  * [25] Eric T Steimle and Michael L Hall.  Unmanned surface vehicles as environmental monitoring and assessment tools.  In OCEANS 2006, pages 1–5. IEEE, 2006.
  * [26] Hugo Ferreira, C Almeida, A Martins, J Almeida, N Dias, A Dias, and E Silva.  Autonomous bathymetry for risk assessment with roaz robotic surface vehicle.  In Oceans 2009-Europe, pages 1–6. Ieee, 2009.
  * [27] Alireza Shojaei, Hashem Izadi Moud, and Ian Flood.  Proof of concept for the use of small unmanned surface vehicle in built environment management.  In Proceeding of Construction Research Congress, pages 148–157, 2018.
  * [28] Abhinav Valada, Prasanna Velagapudi, Balajee Kannan, Christopher Tomaszewski, George Kantor, and Paul Scerri.  Development of a low cost multi-robot autonomous marine surface platform.  In Field and service robotics, pages 643–658. Springer, 2014.
  * [29] Jessica Simmerman Wolfe.  Evaluation of the utility and performance of an autonomous surface vehicle for mobile monitoring of waterborne biochemical agents.  PhD thesis, Mississippi State University, 2021.
  * [30] Bharat Bhushan and Gadadhar Sahoo.  Requirements, protocols, and security challenges in wireless sensor networks: An industrial perspective.  In Handbook of computer networks and cyber security, pages 683–713. Springer, 2020.
  * [31] Paul Mahacek.  Dynamic analysis of a swath vessel.  MBARI Internship Report, pages 1–13, 2005.
  * [32] Md Manjurul Ahsan, MA Mahmud, Pritom Kumar Saha, Kishor Datta Gupta, and Zahed Siddique.  Effect of data scaling methods on machine learning algorithms and model performance.  Technologies, 9(3):52, 2021.
  * [33] Rupal Roy, Maidul Islam, Nafiz Sadman, MA Mahmud, Kishor Datta Gupta, and Md Manjurul Ahsan.  A review on comparative remarks, performance evaluation and improvement strategies of quadrotor controllers.  Technologies, 9(2):37, 2021.


# Leaf Segmentation and Counting with Deep Learning: on Model Certainty, Test-
Time Augmentation, Trade-Offs

Douglas P. S. Gomes
School of Computing and Mathematics
Charles Sturt University
Melbourne, Australia
<EMAIL_ADDRESS>
Lihong Zheng
School of Computing and Mathematics
Charles Sturt University
Wagga Wagga, Australia
<EMAIL_ADDRESS>

###### Abstract

Plant phenotyping tasks such as leaf segmentation and counting are fundamental
to the study of phenotypic traits. Since it is well-suited for these tasks,
deep supervised learning has been prevalent in recent works proposing better
performing models at segmenting and counting leaves. Despite good efforts from
research groups, one of the main challenges for proposing better methods is
still the limitation of labelled data availability. The main efforts of the
field seem to be augmenting existing limited data sets, and some aspects of
the modelling process have been under-discussed. This paper explores such
topics and present experiments that led to the development of the best-
performing method in the Leaf Segmentation Challenge and in another external
data set of Komatsuna plants. The model has competitive performance while been
arguably simpler than other recently proposed ones. The experiments also
brought insights such as the fact that model cardinality and test-time
augmentation may have strong applications in object segmentation of single
class and high occlusion, and regarding the data distribution of recently
proposed data sets for benchmarking.

_Keywords_ challenge $\cdot$ counting $\cdot$ deep learning $\cdot$ leaf
segmentation $\cdot$ plant phenotyping $\cdot$ test-time augmentation

##  1 Introduction

As part of the broader field of plant phenotyping, the tasks of leaf
segmentation and counting have been receiving increasing attention with some
distinguishable work focusing solely on either or both of these tasks
(Kuznichov et al. (2019); Ward and Moghadam (2020); Zhu et al. (2018)). This
is not necessarily surprising since the analysis of phonetic traits can play a
role in the advancement of plant science and in the aspects of breeding and
crop management. Nevertheless, there are potential underlying factors for such
growth besides the importance of the subject. The potentially most relevant
one has to do with possible benefits from applying computer vision-based
techniques to the tasks of plant phenotyping, especially ones in deep
learning, which also experienced tremendous growth in the past years. Such
benefits are mainly composed of moving from manually or semi-automatically
labelling, which can be much onerous, expensive and low-throughput, to fully
end-to-end segmentation (leaf area measuring) and counting in a high-
throughput manner.

These plant-phenotyping tasks relate to broader problems in computer vision:
instance segmentation (leaf segmentation) and object detection (leaf
counting). It is not surprising, therefore, that deep learning has been so
prevalent in works addressing these tasks in plant phenotyping since it has
also been responsible for new benchmarking in their respective computer-vision
tasks. Leaf segmentation can be seen as a particular case of instance
segmentation where the number of classes is equal to one. Leaf counting
relates to object detection in the sense that the number of leaves on an image
can be given by the number of instances detected from such a single-class
object. It is important to note that localization (segmentation) and detection
have an intrinsic trade-off, which is usually regulated by a detection
threshold that influences how many instances are counted as positive
detections and how well-localized they are.

As other maturing fields in this current period of higher accessibility to
supervised-learning algorithms and increasing volume of works, there is a need
for standardized benchmarks and data sets to objectively compare different
solutions. To this issue, one can find remarkable efforts from some research
groups that made their data publicly available (Uchiyama et al. (2017);
Minervini et al. (2015a)). However, as will be further discussed here, all
data sets carry relevant bias and one should make an effort to measure how the
results from one data set will actually generalise to others. Moreover, as the
use of deep learning is relatively recent in this field, there are some under-
discussed topics like the parameters of the model that relates to certainty,
which can have great implications and trade-offs in tasks like segmentation
and instance detection.

Two important data sets available in the literature are named here as the
CVPPP (Minervini et al. (2015a)) and the Komatsuna (Uchiyama et al. (2017)).
The CVPPP is the most popular and firstly introduced in the Computer Vision
Problems in Plant Phenotyping (CVPPP) 2014 workshop. The authors not only
released the well-annotated data set of Arabidopsis and Tobacco images taken
from the top but also specific metrics for benchmarking solutions. Later on,
the team also set up a CodaLab server Scharr (2018) as the Leaf Segmentation
Challenge (LSC), which was a test set where the ground truth labels are not
shared so solutions can be effectively compared in the released metrics. The
second data set comprises images of Komatsuna also taken from the top with
ground truth, firstly presented in the ICCV Workshop on Computer Vision
Problems in Plant Phenotyping in 2017. This data set, as the validation server
in the CodaLab, is used here strictly as a test set to obtain metrics of
generalisation and indirectly measure the bias of the CVPPP data set (used for
learning), which is now a benchmark of the field.

One interesting point to the work proposed here is that past works presenting
competitive results on the CVPPP data set have been achieving it through
ingenious data augmentation techniques (Gomes and Zheng (2020)). Most involve
the generation of synthetic data via elaborate methods ranging from copy-and-
paste methods (Kuznichov et al. (2019)) to generative artificial networks (Zhu
et al. (2018)). Although some are simpler than others, they all represent
expensive techniques which are not obviously worth the investment of time and
resources. The results presented here attest to this fact by presenting models
that rank first in the LSC without using synthetic data generation methods and
accessible computer-vision frameworks. The results are achieved by simple data
augmentation and evaluation techniques, representing an inexpensive
alternative to other more intricate methods. As previously mentioned and
important point, the effect of the detection threshold on the trade-off of
localization and instance detection is also performed. They show that there
are some disadvantages when adopting metrics such as dice score and leaf count
for benchmarking. Moreover, experiments with models of different depth and
cardinality are also performed to attest for their different performance gain
and losses. They are part of the analysis of bias in the CVPPP data set and
are illuminating as to their influence in overfitting and performance on the
tasks of segmentation and instance counting. In short, the following
contributions are proposed:

  * •

Strategies for producing leaf segmentation and counting models presenting
state-of-the-art results while simpler.

  * •

An analysis of the effect of different detection thresholds and its trade-off
in segmentation and counting.

  * •

The effect of model size, cardinality, and test-time augmentation in leaf
segmentation.

  * •

Experiments and commentary on the popular CVPPP test data set regarding its
bias and potential influence in model generalisation.

##  2 Related work

There are some noteworthy works that comprise the same scope as this paper and
should therefore be commented. The field of plant phenotyping is much larger
than such a scope, so related works are going to be the ones considering
similar tasks to the ones considered here, i.e. leaf segmentation and
counting. To make the results as comparable as possible, a focus will be given
to works that use the same data set as adopted in this paper. That does not
necessarily limit the number of works, as the CVPPP has become a benchmark to
these tasks and even used in works proposing instance segmentation generic
techniques (Ren and Zemel (2017)). However, this does limit the works mainly
to the ones related to the CVPPP as only two other papers adopt the Komatsuna
data set, as identified by the present authors. A noteworthy point regarding
recent works presented in the past two to three years is that many share one
particular characteristic: the main contribution and alleged boost in
performance come from data augmentation applications. They range from simple
techniques to intricate pipelines to generate synthetic data to augment
training. The fact that they are so prevalent even inspired the authors of
this paper to write a review describing these methods (Gomes and Zheng
(2020)).

A potentially simple technique for data augmentation, for example, is creating
synthetic data by selecting related backgrounds and pasting cut instances of
plants, or seeds, in random orientations. The work by Kuznichov et al. (2019)
proposes such a technique on the CVPPP data set as a structured collage, in
which the pasting follows specific rules like the centering the leaves in a
pot background to resemble a rosette image taken from the top. Another work by
Toda et al. (2019) presented the same technique in a seed segmentation task,
obtaining significant performance gains.

Other more intricate techniques can involve using Generative Artificial
Network (GANs) or complicated modelling pipelines to generate synthetic data.
As an example of using such an approach, Ubbens et al. (2018) used an
L-systems-based simulator software to model plants with traits variable
according to probabilities distributions. It allegedly presented evidence of
the transferability in learning from plant modelling to the tasks of leaf
counting. Moreover, the authors in Ward and Moghadam (2020) presented a full-
dedicated pipeline comprising background processing, texture generation and
plant assembly. The authors claim to have surpassed state-of-the-art results
at the time, and that their technique would help to bridge the gap between
species. They were also one of the few that submitted their results to the LSC
server and thus more objectively compared. Regarding the methods using GANs to
augment training data (Valerio Giuffrida et al. (2017); Zhu et al. (2018)),
they have been mostly used to address the task of leaf counting. They seem to
be effective at boosting performance but suffer from being complex to
implement and highly specific, focusing on a single plant species from which
images are generated.

Given the number of papers using the approach of data augmentation to address
the tasks of leaf segmentation and counting, one could argue that they are
effective, but they might also be too domain-specific. When augmenting
training data with GANs, for example, the images generated were from
Arabidopsis, and no evidence was provided that it would help with any other
species. The same is true for the works using modelling and collage
augmentation. It is one of the reasons why an external data set was added to
the validation step to the work proposed here. An exception of such critique
is the work by Ward and Moghadam (2020), which had the goal of generalising to
other species with their data generation pipeline; they tested their method
with two data sets, one of which was the one used here. Since they were also
one of the few that submitted their evaluation to the LSC server, it is one of
the works that can be most objectively compared. This point is important
because, as most of these methods are not officially present in the LSC
challenge, the comparison of the results will not be strict and will be
related to the results presented in their paper. Often such results are
extracted from splits made in the CVPPP data set from A1-A4 into training and
testing sets. Hence, the results presented here, which were extracted from a
test set in the server with no ground truth available, should receive a high
weight regarding validation and generalisation ability. One honourable mention
is the recent method by Wu et al. (2020), which presents a whole new
architecture based on the U-Net and clustering pixels with the same
characteristics to obtain leaf masks. Nevertheless, the results presented here
also overperform their method in the leaf segmentation task, like all other
mentioned works.

##  3 Data and methods

This work presents experiments testing hypotheses around the tasks of leaf
segmentation and counting and proposes techniques that result in a computer
vision model achieving results comparable to the state of the art in such
tasks. This section describes the two independent data sets, the experiments
performed to assess the hypotheses, and the techniques that led to the
competitive results discovered throughout the investigation on these tasks.

###  3.1 Data sets

The analysis proposed here is based primarily on the CVPPP data set while the
validation of the presented hypotheses is performed on the Leaf Segmentation
Challenge server and on an independent similar data set of a different plant
species (Komatsuna). As previously noted, the importance of the CVPPP is given
by its adoption in many works in the field of plant phenotyping, especially
regarding the problems of leaf segmentation and counting. The challenge server
then becomes an interesting validation tool as it comprises a validation set
that no one has access to the ground-truth masks, although not many authors
submit their results there for comparison to other works. Nevertheless, as
some of the insights presented here are comments on the data distribution of
the CVPPP and its effect on the tasks of segmentation and counting, an
external data set is needed to test and validate the hypothesis.

####  3.1.1 The CVPPP data set and challenge

The CVPPP data set represents perhaps that largest effort to address the
ongoing problem of lacking benchmark data sets and metrics for specific tasks
such as leaf segmentation and counting on a controlled environment. First
presented in 2014, but updated in 2017, the data set comprises images of
mainly Arabidopsis and a small portion of Tobacco plants and can be accessed
by Minervini et al. (2015b). The authors stated that the images have been
collected from several sites from growth chamber experiments, and it is
divided into four groups named A1 to A4. There are 810 images with ground-
truth masks available, from which 783 are from Arabidopsis and 27 from
Tobacco, showing that the later represents only a small part of that data.
These images are all taken from the top, probably as an attempt to mitigate
occlusion, as illustrated in Fig. 1.

(a) Examples of plants from the CVPPP dataset.

(b) Examples of possible labels.

Figure 1: Examples of plant images in the CVPPPP dataset.

The organizers of this data set made the critical decision of creating a
separate data set, A5, and uploading it to an evaluation server in CodaLAB
labelled as the Leaf Segmentation Challenge (LSC). They also set up metrics
for the tasks trained on the data set, from which two central are adopted in
this paper: Symmetric Best Dice (SBD) for segmentation and Absolute Difference
in Count (abs. DiC) for counting. These efforts paid off with the data set now
being used as a benchmark in many recent papers (Ren and Zemel (2017);
Kuznichov et al. (2019); Ward and Moghadam (2020); Zhu et al. (2018)).
Nevertheless, although many authors have been using it as a training and test
set, a few have submitted to the challenge server. The model presented in this
paper, which has used all A1-A4 images in training and tested in the A5
(server), currently ranks first in the challenge leaderboard.

####  3.1.2 Komatsuna data set

In a similar manner to the CVPPP data set, the Komatsuna (Uchiyama et al.
(2017)) comprises images of plants taken from the top but now on a different
species, which is hoped to help attest generalization. Although the data set
was conceptualized to help address 3D phenotyping problems, it is mainly used
here as a test set only due to its similarities to the CVPPP (training set),
as illustrated in Fig. 2. Such a test set is composed of 271 images, and the
evaluation is performed in the same metrics as used in the CVPPP testing. One
final feature of this data set is that the images contain plants in many
different stages of growth, as the data set was also conceptualized with this
goal in mind.

(a)

(b)

Figure 2: Examples of a plant image in the Komatsuna data set (a) and its
ground-truth masks (b).

###  3.2 Detection threshold effect

The detection threshold is a fundamental part of a classification or decision-
making system and can have severe effects on its output. This effect is more
explicitly illustrated by the Receiver Operating Characteristic (ROC) curve,
which plots the false versus true positive rate. It shows that there is a
trade-off between the number of detection a system makes and how many are
actually true. For example, one could trivially have a maximum true positive
rate by just classifying all examples as positive, but that would also lead to
a very large false-positive rate. In computer vision, this concept is often
discussed in terms of precision and recall. The effect that it has on object
detection is remarkable: if one sets a low detection threshold, many
candidates of objects will be selected, but a larger number will be miss
detections or have worse localization (e.g. bounding boxes). This is mainly a
problem of certainty in the decision-making system and previously referred to
in the literature as cost-sensitive analysis (Elkan (2001)).

Previous works proposing algorithms for segmenting leaves on the CVPPP have
been using mainly deep learning, but also a specific method for computer
vision called Mask R-CNN (He et al. (2017)). Such a method contrasts with
other object detection algorithms for using a two-step process rather than
being one-shot like methods such as YOLO (Redmon et al. (2016)). For the
particular case on the Mask R-CNN framework He et al. (2017), the trade-off
mentioned in the last paragraph can come into play in the mask proposal head,
which is the addition to the classic Faster R-CNN architecture. This head
produces a fixed-sized mask for every region of interested created in the
first stage, and the detections are attested when the intersection over union
(IoU) value is higher than an arbitrary threshold. Therefore, such an effect
is relevant to the work presented here since recent methods for leaf
segmentation often does not discuss the effect that different thresholds can
have on the final output. To address such a gap, experiments with different
thresholds were performed here and analysed in the Mask R-CNN framework. The
results reveal a significant impact in the dice score when segmenting leaves,
attesting to the hypothesis that the discussion about the detection threshold
should not be neglected in works addressing this task. Simple changes boosted
the performance of simple methods to state-of-the-art results but at the
expensive trade-off with the leaf counting metric. Moreover, these experiments
also bring relevant insights to the data sets used when results from the CVPPP
and Komatsuna are compared, especially regarding their distribution and
potential for overfitting.

###  3.3 Model depth and cardinality

Although it is somewhat expected that deeper models often lead to better
performance, it is important to test such hypotheses, especially when the size
of these data sets are put in context. The CVPPP data set, used here for
training, comprises less than a thousand images, hence arguably prone to
overfitting and the reason why so many previous works have been proposing data
augmentation techniques with it. To test the effect of model depth, two
popular variations of the Mask R-CNN are tested here, one having a 50-layer
backbone while the other having 101 layers.

To make the experiments potentially more insightful, not only two different
depths were compared but also a variation with different cardinality on the
101-layer backbone was added. This addition resulted in two models having a
ResNet backbone (He et al. (2016)), one with 50 and the other with 101 layers,
and one model having a ResXNet backbone (Xie et al. (2017)). The difference
between these two types of backbone is in the size of transformations, which
the authors call cardinality (Xie et al. (2017)), but they do have the same
dimensions regarding depth and width. The results from such comparison are
much less trivial than expected, with the ResXNet performance pointing to the
fact that it may have significant advantages to its predecessor (ResNet) while
dealing with smaller data sets to what concerns generalization and
overfitting.

###  3.4 Test-time augmentation

As previously alluded to, the experiments with the detection threshold were
revealing and conducive to better performance on the leaf segmentation but at
a cost to the leaf counting task. That may be sufficient to attest state-of-
the-art results on the segmentation task, but as the results attested, the
performance of rate of positive detection (counting) comes close to being
relatively insufficient to be considered a solution. Nevertheless, just
because such trade-offs exist, it does not necessarily mean that it can’t be
mitigated.

The solution proposed here is to apply a technique called Test-Time
Augmentation (TTA) to mitigate the effects that a different detection
threshold causes in the leaf counting task while preserving or even boosting
the leaf segmentation performance. In short, test-time augmentation comprises
the process of performing inference on the original image and different
augmented versions of it. The predictions are then averaged through a pixel-
wise voting system and the final output is composed of averaged masks of each
instance. As theoretically formulated by Wang et al. (2019), the key reason
for why it works is that TTA helps to eliminate overconfident incorrect
predictions. This is particularly important to the problem of leaf
segmentation and counting because it comprises a problem with high occlusion
of objects, meaning a combination of a leaf on top of the other is usually
classified as a single instance. This is the reason why most results in the
challenge leaderboard present a negative difference in the count; they are
predicting fewer leaves than are actually present in the images. As an
additional note, it is important to highlight that TTA does not influence the
process of training in contrast to usual data augmentation; it is only present
in the inference part of the model evaluation.

The application of TTA here is composed of definite parts: (i) augmentation of
the original image, (ii) inference, (iii) reversing the augmentation on the
predictions, (iv) aligning instances of leaves, and (v) averaging the
instances masks. The augmentation of leaves is composed of four, simple ones:
horizontal and vertical flipping, and 90 degrees rotation clockwise and
anticlockwise. This process results in five independent inferences for each
image (original plus four augmented versions). After predictions masks on each
of these versions are made, they need to be disaugmented so they can be
averaged with the original one. The most challenging part, nevertheless, is
that the model is agnostic to the versions of the images so the instance masks
in each version are not ordered in the same manner. Hence, an ordering process
takes place, which tries to relate the same instance of leaf for all versions
of the original image. This is performed by measuring IoU between every
instance of the version with the most number of total instances and other
versions of the original image. Instances with a value higher than 0.5 are
considered the same instance and thus put in the same order for averaging. The
fact that the version with the most number of total instances is used in the
comparison makes the system to be biased to the version less likely to have
committed the error of considering two occluded instances as one. The
averaging is somewhat straightforward in the sense that it is a pixel-wise
majority voting process; if the mask pixel is positive for a particular
instance in more versions that not, the pixel is considered to be existent.
Although such a procedure adds processing time to the inference of an image,
it is arguably simpler than other methods that need deep technical knowledge
to model plants to create synthetic data for training augmentation. The
results also point to the fact that it is a reasonable solution for the object
occlusion problem leading to expressive results, which might make the
performance time trade-off worth it.

###  3.5 Programming environment and frameworks

All the methods described here were performed using accessible and mainly free
and open-source frameworks. Regarding the hardware, for example, the Google
Colab platform was used. It currently has some time restrictions but offers
GPUs of up to 16 GB in memory for free at a limited but considerable amount of
time. Colab also has a convenient integration with Google Drive that allows
mounting and storage of the data for training. Software-wise, the modelling
and training libraries used were part of the Detectron II (Wu et al. (2019))
framework by Facebook, which is a well-maintained, open-source library with
pre-trained models available. All the models trained started with pre-trained
weights from the COCO data set; such models’ weights can be found in the
Detectron II’s model zoo. The training schedule followed most of the default
parameters but with the learning rate set to 0.0001. Checkpoints of the models
were saved at every 5,000 interactions and trained until stop converging,
which was around 70-100,000 iterations for most. The mainly two augmentation
strategies used were horizontal flipping and rotation; for the 101-layer
models, the augmentations were random cropping and rotation since flipping the
images online consumes much extra memory. The results showed that this did not
make a considerable difference while evaluating the models.

##  4 Results

Two points are worth noting before the presentation of the results of the
experiments described in the last section. The first concerns results
presented regarding the CVPPP data set. They were extracted from the ‘Leaf
Segmentation Challenge’ server on CodaLab (Scharr (2018)), and the authors do
not have access to the ground truth of this test set. Regarding the Komatsuna
data set, it was only used for testing and took no part in the training of the
models. The second point relates to how the results and discussions are going
to be presented. Since they are from well-defined, independent experiments,
the discussions are going to be presented in this section, following each
experiment rather than having results and discussions presented in separate
sections.

###  4.1 Threshold and certainty

The effect of the detection threshold is expressive in both the CVPPP and
Komatsuna test sets. For the CVPPP, Fig. 3 is set to illustrate the comparison
between the performance of three key thresholds in the metrics of leaf
segmentation (SBD) and counting (abs. DiC). What is most interesting about
this comparison is the gains in performance that one makes from a 0.5 to 0.7
threshold. The authors of this paper could not find any discrimination on the
previous recent works using deep learning regarding such a threshold, except
for Ward and Moghadam (2020), which state to use the standard value of 0.5.
The performance gains from going to a 0.7 threshold are so strong that would
make a method from ranking low on the leaderboard to one being among the top
five. These gains are also strong when compared to the works in the literature
that had not published their results as submission in the LSC (Gomes and Zheng
(2020)). Moreover, going from a threshold of 0.7 to 0.9 would make a method to
be in the first place in the leaf segmentation leaderboard but, in this case,
with a higher cost to the leaf counting metric. It is interesting to note that
the performance gains in the SBD metric are mostly at the cost of a worse
performance in leaf counting. That is not necessarily surprising given the
previously mentioned points about the trade-off between false and true
positive rates, but it is in the sense that such results were achieved by a
simple, standard application of Mask R-CNN only using flip and rotation as
augmentation strategies. However, one can also notice from Fig. 3 that the
gains are not linear; a strong gain in the SBD metric from 0.5 to 0.7 does not
cost as much in the abs. DiC metric as the move from 0.7 to 0.9.

Figure 3: Comparison of the effect of different detection thresholds on the
CVPPP test set in the leaf segmentation metric (SBD) and counting (absolute
DiC).

While testing different thresholds on the Komatsuna data set, however, results
showed that the performance gains of the different thresholds might be
somewhat overly expressed in the CVPPP data set. Fig. 4 is set to illustrate
the difference in the SBD and DiC in detail. The results show that although
the detection threshold can have a great influence on such an external data
set, the leaf segmentation metric does not monotonically increase to from 0.5
to 0.9, as seen in the CVPPP. This fact is remarkably insightful, pointing to
the fact that the CVPPP might represent a narrow data distribution that can
greatly affect model generalisation. In simpler terms, the SBD monotonically
increases due to the fact that the examples in the test set are too similar to
the ones in the training set. It should thereby raise some concerns to what it
means to optimize an object segmentation metric on the CVPPP data set, which
has become somewhat of a benchmark in this field, in relation to other data
sets for the same task. The best SBD achieved in the Komatsuna data set was
0.7452, with the model obtaining an abs. DiC of 0.9594 at a threshold of 0.6.
This result is also noteworthy since it surpasses the one achieved by the
previous work with the highest SBD score, at the best of our knowledge, which
was 0.7169 (Ward and Moghadam (2020)), when no images from the data set (out
of context) are used.

Figure 4: The effect of different thresholds on the Komatsuna test set in the
SBD and abs. DiC metrics.

###  4.2 Model depth and cardinality

A consequent question is how models with different depth will perform in these
tasks while trained by the same algorithm. The results from such an experiment
were also revealing; Fig. 5 is set to illustrate some comparisons. In this
case, the figure compares the 50- and the two 101-layer models (ResNet and
ResXNet), all for the same threshold of 0.7. The first image (a) illustrate
that there are small differences from the models of 50 and 101 layers with the
ResNet backbone regarding the segmentation but a performance gain in the leaf
counting task. In advantage, the ResXNet backbone seems to have significant
gains in both segmentation and count. However, the Komatsuna data set can be
used again to clarify the validity of the potential gains. In the second image
(b), it is clear that the ResNet backbone with 101 layers actually losses a
lost of performance in both tasks, while little difference is seen on the
ResXNet. This comparison reveals important insights regarding this
characteristic called cardinality, which in this case, stopped the deeper
model to overfit as it happened with the ResNet of the same depth. It points
to the fact that such characteristic — the only difference between these
backbones — may work better in problems of objects with high occlusion,
allowing the model to generalise significantly better to other data sets.
Lastly also pointing to the mentioned fact that the data distribution from the
CVPPP might be too narrow to result in strong generalisation in other data
sets, as any model can easily overfit to it and perform worse when tested on
external data sets.

(a)

(b)

Figure 5: Comparison of performance of models of different depth and
cardinality on the tests sets: a) CVPPP and b) Komatsuna. Detection threshold
= 0.7.

###  4.3 Test-time augmentation

The test-time augmentation hypothesis is proposed here as a way to alleviate
the trade-off or maximize the relationship between the segmentation and
counting scores. A classifier that is able to segment leaves but only does it
for the ones that it has a high certainty of detection is not as useful,
despite higher SBD score. To that end, the experiment showed to be consistent
for both data sets but it showed to be more useful for ResXNet backbones as
illustrated in Fig. 7. The model 101-layer ResXNet model, which is currently
submitted in the LSC, have achieved approximately 2% higher performance in the
SBD metric than other candidates while having a more reasonable counting
performance than other thresholds that achieve comparable performance but do
not use TTA. To illustrate the effect that such a technique can have on
segmenting the masks, Fig. 6 shows examples of segmentation instances where
the effects of using TTA are expressive.

(a) Single instance mask.

(b) All instances.

(c) All instances.

Figure 6: Examples of instances where there was an expressive difference from
not using (left) and using (right) TTA.

This result becomes more important when the Komatsuna is considered, which
most methods in the literature do not take into account. As shown in Fig. 3,
this performance in the SBD metric can be achieved by forcefully increasing
the threshold of detection. However, as shown in Fig. 4, this result does not
translate to the external data set (Komatsuna), which presents very poor
performance for a higher threshold with no TTA. Therefore, by applying TTA,
the model not only makes a better compromise between segmentation and counting
on the CVPPP data set but also generalises much better in the Komatsuna data
set. It achieves the highest result observed in both segmentation and counting
metrics on this external data set with an SBD of 0.7553 and an abs. DiC of
0.881 (higher than the CVPPPP — the test set closely related to the training
set).

(a)

(b)

Figure 7: TTA performance comparison in the 50-layer ResNet backbone and the
101-layer ResXNet backbone on a) the CVPPP and b) Komatsuna data set.

###  4.4 Comparison with previous and LSC methods

The analyses and results presented attest that there are different ways to
present a model, especially when considering the trade-off between the tasks
of segmentation and counting. Nevertheless, given the strategies used here for
mitigating the costs of such trade-off, the resulting models present
competitive results for both tasks despite the bias adopted. Table 1, for
example, shows the leaderboard of the LSC when sorted by the SBD metric. When
biased towards segmentation (higher threshold), the model performs 2% higher
than the second place while demonstrating competitive scores for leaf
counting. If the threshold is biased towards the task of counting, however,
the model can tie the results with the best-performing model in the abs. DiC
metric while also presenting the same score for SBD as the best submission for
the SBD metric. As illustrated in Table 2, the best submission for leaf
counting have lower performance at segmenting the leaves than the one proposed
here. It should be noted that there is an advantage in tying when the model
proposed here does not use any synthetic and additional data in training and
uses a straightforward Mask R-CNN algorithm. The comparison of the Komatsuna
data set with another method is presented in Table 3. It is expected that, as
this is a relatively recent data set, more works will present their results
using this data set as a benchmark. It is also worth remembering that the
Komatsuna data set was not used at any point while training the model but only
while testing it. Finally, examples of inferences in the CVPPP and Komatsuna
data sets are illustrated in Fig. 8.

Table 1: Top 5 LSC leaderboard sorted by SBD score (segmentation biased) Name (user) | SBD | |DiC|
---|---|---
dougpsg (ours, segment. biased) | 0.90 | 1.50
LfB | 0.88 | 3.97
wuzi | 0.87 | 0.9
yukinohana | 0.87 | 0.89
looooong | 0.86 | 1.24
Table 2: Top 5 LSC leaderboard sorted by |DiC| score (count biased) Name (user) | |DiC| | SBD
---|---|---
yukinohana | 0.89 | 0.87
dougpsg (ours, count biased) | 0.89 | 0.88
wuzi | 0.9 | 0.87
looooong | 1.24 | 0.86
StevenYuan | 2.74 | 0.76
Table 3: Komatsuna comparison Work | SBD | |DiC|
---|---|---
Ours, segment biased | 0.75 | 0.88
Ward and Moghadam (2020) | 0.72 |

(a)

(b)

Figure 8: Examples of inference on the images from the (a) CVPPP and (b)
Komatsuna data sets.

##  5 Conclusions

This paper presents a series of experiments, which culminates on a model
achieving state-of-the-art results for the task of leaf segmentation while
presenting competitive results for leaf counting. At the same time,
differently from most recent methods in the related literature, such results
are achieved without complicated pipelines for synthetic data generation that
attempt to augment the limited CVPPP training set. While built and trained on
an accessible platform without significant modifications to the standard Mask
R-CNN framework, this model represents a simpler solution that achieves best
results in both the LSC and on the external data set (Komatsuna). Most
insights came from addressing under-discussed topics such as the effect of the
detection threshold on the task at hand and ways to mitigate the trade-off
resulted from it. The best results came from a deeper model (101 layers) with
different cardinality (ResXNet), which could use the benefits of performance
gains and trade-off mitigation given by test-time augmentation. One can argue
that these aspects are significantly relevant to tasks like leaf segmentation,
which are composed of objects of single class that suffer from high occlusion.
The experiments also generated additional insights about the data set,
confirming potential concerns that the segmentation and counting metrics
should not be considered alone, as they can be forced by changes in the
threshold, and that using data sets of narrow distributions to benchmark
solutions may lead to erroneous conclusions regarding the generalisation of
the resulted models.

## References

  * Kuznichov et al. [2019] Dmitry Kuznichov, Alon Zvirin, Yaron Honen, and Ron Kimmel.  Data augmentation for leaf segmentation and counting tasks in rosette plants.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops_ , pages 0–0, 2019.
  * Ward and Moghadam [2020] Daniel Ward and Peyman Moghadam.  Scalable learning for bridging the species gap in image-based plant phenotyping.  _Computer Vision and Image Understanding_ , page 103009, 2020.
  * Zhu et al. [2018] Yezi Zhu, Marc Aoun, Marcel Krijn, Joaquin Vanschoren, and High Tech Campus.  Data augmentation using conditional generative adversarial networks for leaf counting in arabidopsis plants.  In _BMVC_ , page 324, 2018.
  * Uchiyama et al. [2017] Hideaki Uchiyama, Shunsuke Sakurai, Masashi Mishima, Daisaku Arita, Takashi Okayasu, Atsushi Shimada, and Rin-ichiro Taniguchi.  An easy-to-setup 3d phenotyping platform for komatsuna dataset.  In _Proceedings of the IEEE International Conference on Computer Vision Workshops_ , pages 2038–2045, 2017.
  * Minervini et al. [2015a] Massimo Minervini, Andreas Fischbach, Hanno Scharr, and Sotirios A. Tsaftaris.  Finely-grained annotated datasets for image-based plant phenotyping.  _Pattern Recognition Letters_ , pages –, 2015a.  ISSN 0167-8655.  doi:http://dx.doi.org/10.1016/j.patrec.2015.10.013.  URL http://www.sciencedirect.com/science/article/pii/S0167865515003645.
  * Scharr [2018] H. Scharr.  Leaf segmentation challenge (lsc), 2018.  URL https://competitions.codalab.org/competitions/18405.
  * Gomes and Zheng [2020] Douglas P S Gomes and Lihong Zheng.  Recent data augmentation strategies for deep learning in plant phenotyping and their significance, Aug 2020.  URL engrxiv.org/t3q5p.
  * Ren and Zemel [2017] Mengye Ren and Richard S Zemel.  End-to-end instance segmentation with recurrent attention.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 6656–6664, 2017.
  * Toda et al. [2019] Yosuke Toda, Fumio Okura, Jun Ito, Satoshi Okada, Toshinori Kinoshita, Hiroyuki Tsuji, and Daisuke Saisho.  Learning from synthetic dataset for crop seed instance segmentation.  _BioRxiv_ , page 866921, 2019.
  * Ubbens et al. [2018] Jordan Ubbens, Mikolaj Cieslak, Przemyslaw Prusinkiewicz, and Ian Stavness.  The use of plant models in deep learning: an application to leaf counting in rosette plants.  _Plant methods_ , 14(1):6, 2018.
  * Valerio Giuffrida et al. [2017] Mario Valerio Giuffrida, Hanno Scharr, and Sotirios A Tsaftaris.  Arigan: Synthetic arabidopsis plants using generative adversarial network.  In _Proceedings of the IEEE International Conference on Computer Vision Workshops_ , pages 2064–2071, 2017.
  * Wu et al. [2020] Yuli Wu, Long Chen, and Dorit Merhof.  Improving pixel embedding learning through intermediate distance regression supervision for instance segmentation.  _arXiv preprint arXiv:2007.06660_ , 2020.
  * Minervini et al. [2015b] M. Minervini, A. Fischbach, H. Scharr, and S.A. Tsaftaris.  Plant phenotyping datasets, 2015b.  URL http://www.plant-phenotyping.org/datasets.
  * Elkan [2001] Charles Elkan.  The foundations of cost-sensitive learning.  In _International joint conference on artificial intelligence_ , volume 17, pages 973–978. Lawrence Erlbaum Associates Ltd, 2001.
  * He et al. [2017] Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick.  Mask r-cnn.  In _Proceedings of the IEEE international conference on computer vision_ , pages 2961–2969, 2017.
  * Redmon et al. [2016] Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi.  You only look once: Unified, real-time object detection.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 779–788, 2016.
  * He et al. [2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 770–778, 2016.
  * Xie et al. [2017] Saining Xie, Ross Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He.  Aggregated residual transformations for deep neural networks.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 1492–1500, 2017.
  * Wang et al. [2019] Guotai Wang, Wenqi Li, Michael Aertsen, Jan Deprest, Sébastien Ourselin, and Tom Vercauteren.  Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networks.  _Neurocomputing_ , 338:34–45, 2019.
  * Wu et al. [2019] Yuxin Wu, Alexander Kirillov, Francisco Massa, Wan-Yen Lo, and Ross Girshick.  Detectron2.  https://github.com/facebookresearch/detectron2, 2019.