# Time-optimal control of cranes subject to container height constraints

Filipe Marques Barbosa and Johan Löfberg This work was supported by VINNOVA
Competence Center LINK-SIC.F. Marques Barbosa and J. Löfberg are with the
Division of Automatic Control, Department of Electrical Engineering, Linköping
University, Linköping, Sweden<EMAIL_ADDRESS><EMAIL_ADDRESS>10.23919/ACC53348.2022.9867816

###### Abstract

The productivity and efficiency of port operations strongly depend on how fast
a ship can be unloaded and loaded again. With this in mind, ship-to-shore
cranes perform the critical task of transporting containers into and onto a
ship and must do so as fast as possible. Though the problem of minimizing the
time spent in moving the payload has been addressed in previous studies, the
different heights of the container stacks have not been the focus. In this
paper, we perform a change of variable and reformulate the optimization
problem to deal with the constraints on the stack heights. As consequence,
these constraints become trivial and easy to represent since they turn into
bound constraints when the problem is discretized for the numerical solver. To
validate the idea, we simulate a small-scale scenario where different stack
heights are used. The results confirm our idea and the representation of the
stack constraints become indeed trivial. This approach is promising to be
applied in real crane operations and has the potential to enhance their
automation.

##  I INTRODUCTION

Shipping containers revolutionized the transportation of goods across the
globe. Before its introduction, the process of loading and unloading a ship
was extremely slow and they had to stay for weeks in the port at a time.
Thanks to containerization, the productivity and efficiency of ports have been
remarkably enhanced. Moreover, an important performance indicator of a port is
the speed at which a ship is unloaded and loaded again [1, 2].

With this in mind, the efficiency of port operations deeply depends on the
performance of the ship-to-shore (STS) cranes. They are responsible for the
critical task of loading and safely unloading the ship [1, 2, 3]. See Fig. 1.
Thus making their operations faster is crucial for the performance indicators
of the port.

Because of this, various approaches have been proposed to address the faster
loading and unloading of containers. Some authors have focused on minimizing
the swing and other oscillations to minimize the time of the loading and
unloading process. For instance, in the works done by Kreuzer et. al., [1] and
Kim et. al., [4], where the main idea is that a good estimation of the sway
angle will make it easier to be mitigated and thus lead to faster operations.
In the former, an unscented Kalman filter is designed to provide an accurate
estimation of the payload swing. Then, it is validated by applying it in
different control techniques. In the latter, an anti-sway controller is
designed without the use of a vision system, thus making its implementation
cheaper.

Following this, optimal control has also been used in crane operations as a
combination of both anti-sway and trolley motion control. Sakawa and Shindo
[2] divide the payload motion into five sections and derive an optimal speed
reference trajectory, and the swing of the payload is then minimized.
Furthermore, Maghsoudi et al., [5] have used a three-dimensional model of the
crane and tuned a PID controller by using a cost function with the aim of fast
payload positioning and minimum sway.

Moreover, time-optimal control has been used as a natural approach when one
aims to make the loading and unloading as fast as possible. In such
approaches, the objective is to minimize the final time $t_{f}$. That is, the
total time for the payload to go from an initial position at rest to a desired
final position at rest again.

With this in mind, Auernig and Troger [6] use the Pontryagin maximum principle
and obtain the minimum-time solution for moving the payload. Also, based on
differential flatness Chen et al., [7] propose a time-optimal offline
trajectory planning method and a tracking controller.

Furthermore, time-optimal control has also been combined with anti-sway. Al-
Garni et al., [8] use a nonlinear dynamic model and constructs a cost function
according to a performance index. This performance index is measured under the
condition of making the process of loading or unloading in a minimum time as
well as reducing the payload sway. Da Cruz and Leonardi [9] use linear
programming to solve the minimum-time anti-sway motion problem. The solution
is obtained by solving a sequence of fixed-time maximum-range problems.

Those all are, without doubt, valuable contributions to the field of control
of overhead cranes. However, a problem with much of the literature is that
they do not address the different heights that the container stacks can have
along the path from the shore to a specific position on the ship. As mentioned
before, some works focus on estimating the sway in order to reduce it and
consequently the transferring time e.g., [1] and [4]. Thus, hoisting movements
are naturally not addressed. Furthermore, some works that explicitly deal with
minimum time do not address hoisting at all. A constant length of the rope is
used in [5] and [7], and it is considered given in [9]. Moreover, works
dealing with hoisting do not take stack heights into account e.g., [2], [6]
and [8].

In this paper, we propose a time-optimal control approach to address the
problem of loading and unloading a ship in the minimum time. Additionally, the
different and arbitrary heights that the container stacks can have are taken
into account here. Moreover, the novelty of this work is an easy and intuitive
way to represent the stack heights into the optimization problem. This is made
possible through a change of variable and reformulation of the optimal control
problem. After the variable change, time $t$ becomes a state variable and the
payload coordinate along the horizontal axis $x_{p}$ becomes the free
variable. Now with the payload position as the free variable instead of time,
the container avoidance constraints will turn into bound constraints when the
problem is discretized for the numerical solver. This means that no functional
representation of the stack heights is required.

This paper is organized as follows: Section II presents the nonlinear model
used to describe the crane movements and the original problem formulation,
with time as the free variable. Which has been the standard way to solve this
problem. Section III brings the variable change and the optimal control
problem reformulation. Section IV shows how the stack heights are now
represented along the horizontal axis, leading to a form that is suitable for
numerical optimization. Section V depicts an example to illustrate the
proposed idea and subsequently discuss it. Lastly, Section VI brings the
conclusion of the work.

Figure 1: A ship-to-shore crane schematic.

##  II MODELLING AND ORIGINAL PROBLEM FORMULATION

In order to control an STS-crane and make the process of loading and unloading
the ship as fast as possible, a model description of its dynamics is obtained
and a time-optimal control problem solved. In this section, we first present a
lumped mass model, following the single-rope hoisting mechanism. Subsequently,
the standard formulation for time-optimal control of this system is presented.

###  II-A Modelling

A simplified representation of the dynamics of overhead cranes can be made by
considering the problem as cart-pendulum with hoisting. To this end, consider
the position of the trolley $x(t)$, the sway $\theta(t)$, the length of the
hoisting rope $l(t)$ and, the payload coordinates $x_{p}(t)$ and $y_{p}(t)$ as
generalized coordinates. The forces $F_{t}(t)$ and $F_{h}(t)$ are the control
inputs applied to the trolley and for hoisting the payload, respectively. A
two-dimensional schematic of this representation showing the general
coordinates and the forces involved in the problem is depicted in Fig. 2.

Figure 2: Two-dimensional schematic representing the trolley and payload
motions.

Similarly to the model obtained in [4] and [10], and additionally including
the position of the payload in general coordinates, the following equations of
motion are obtained

$(m_{1}+m_{2})\ddot{x}(t)+m_{2}l(t)(\ddot{\theta}(t)\cos(\theta(t))-\dot{\theta}^{2}(t)\sin(\theta(t)))+\\\
m_{2}\ddot{l}(t)\sin(\theta(t))+2m_{2}\dot{l}(t)\dot{\theta}(t)\cos(\theta(t))=F_{t}(t)$
(1) $m_{2}\ddot{l}(t)-m_{2}l(t)\dot{\theta}^{2}(t)-m_{2}g\cos(\theta(t))+\\\
m_{2}\ddot{x}(t)\sin(\theta(t))=-F_{h}(t)$ (2)
$l(t)\ddot{\theta}(t)+2\dot{l}(t)\dot{\theta}(t)+g\sin(\theta(t))+\ddot{x}(t)\cos(\theta(t))=0$
(3) $x_{p}(t)=\sin(\theta(t))l(t)+x(t)$ (4) $y_{p}(t)=\cos(\theta(t))l(t).$
(5)

Here $g$ is the acceleration of gravity, and $m_{1}$ and $m_{2}$ are the
trolley and payload masses, respectively. The dynamics of the bridge along the
$i\text{-axis}$ is represented in (1), the hoist motion of the payload in (2),
the sway motion of the payload in (3), and finally the payload position along
$i\text{ and }j\text{-axis}$ are represented in (4) and (5), respectively.
Note that the dynamics of the actuators themselves are not considered, and
this model is formulated considering the dynamics of an overhead crane with a
single hoisting rope mechanism. Moreover, the payload is treated as a point
mass.

Now, choosing the state variables as $x_{1}=x_{p}$, $x_{2}=\dot{x}_{p}$,
$x_{3}=y_{p}$, $x_{4}=\dot{y}_{p}$, $x_{5}=l$, $x_{6}=\dot{l}$, $x_{7}=\theta$
and $x_{8}=\dot{\theta}$, the nonlinear state-space representation is derived
and can be written as:

$\displaystyle\dot{x}_{1}$ $\displaystyle=x_{2}$ (6)
$\displaystyle\dot{x}_{2}$ $\displaystyle=-(u_{2}\sin(x_{7}))/m_{2}$
$\displaystyle\dot{x}_{3}$ $\displaystyle=x_{4}$ $\displaystyle\dot{x}_{4}$
$\displaystyle=-(u_{2}\cos(x_{7}))/m_{2}+g$ $\displaystyle\dot{x}_{5}$
$\displaystyle=x_{6}$ $\displaystyle\dot{x}_{6}$
$\displaystyle=x_{5}x_{8}^{2}+g\cos(x_{7})-u_{2}/m_{2}-\sin(x_{7})(u_{1}+u_{2}\sin(x_{7}))/m_{1}$
$\displaystyle\dot{x}_{7}$ $\displaystyle=x_{8}$ $\displaystyle\dot{x}_{8}$
$\displaystyle=-(2x_{6}x_{8}+g\sin(x_{7})+\cos(x_{7})(u_{1}+u_{2}\sin(x_{7}))/m_{1})/x_{5}$

with the control inputs $u_{1}=F_{t}$ and $u_{2}=F_{h}$.

With the equations of the motion of the crane written in the state-space form,
the time-optimal control problem can be formulated.

###  II-B Original problem formulation

The problem of moving the payload from the initial to the final position in
the minimum time can be expressed as a time-optimal control problem. This way,
we need to minimize a cost function that accounts for the total time spent
transporting the payload. This is done by obtaining a control law that
satisfies the lower and upper bounds of the decision variables and control
inputs.

$\displaystyle\underset{}{\text{minimize}}$ $\displaystyle
T=\int_{0}^{t_{f}}\,dt$ (7) subject to
$\displaystyle\dot{x}(t)=f(t,x(t),u(t))$ $\displaystyle x(0)=x_{0}$
$\displaystyle x(t_{f})=x_{t_{f}}$ $\displaystyle g(x(t))\leq x(t)\leq
f(x(t))$ $\displaystyle u_{min}(t)\leq u(t)\leq u_{max}(t).$

However, as mentioned before, STS-crane operations are usually subject to
geometric constraints, corresponding to the height of the container stacks.
Such constraints are usually nonlinear and non-smooth functions of space.
Because of this, issues arise in formulating and solving the numerical
optimization problem starting from (7).

The most immediate comes from our necessity of constructing the constraints on
the container heights. This means that the geometric constraints in the
numerical model need to be constructed from a function $g(x_{p}(t))$. Due to
the discretization of the problem, the required height that the payload needs
to be at a certain time instance $t^{k}$ is a function $g(x_{p}(t^{k}))$.
Please note that in this paper we use superscript $k$ to indicate
discretization. In this way, the function $g(x_{p}(t))$ needs to be defined
from the current configuration using either continuous approximation or exact
integer programming representation. Moreover, any nontrivial configuration of
container heights will lead to non-convex constraints.

Furthermore, though the objective function looks simple, it cannot be directly
used in a numerical optimization scheme. Once again, the problem here comes
due to the discretization. Since the free variable is also the one being
optimized, the solution will be influenced by the fixed time sampling rate, or
fixed number of control intervals. Thus, it requires a strategy to be used,
for instance, time-elastic bands, presented in [11].

In order to circumvent these problems, we reformulate the optimization problem
through a change of variable.

##  III PROBLEM REFORMULATION

As mentioned in the previous section, container avoidance constraints are
problematic when the independent variable is time $t$. That is, the stack
heights are difficult to be represented when using time as the variable to be
discretized. However, they can be easily described in the position along the
$i\text{-axis}$.

With this in mind, the optimization problem is reformulated in an attempt to
alleviate the above-mentioned issues. This is done by reparametrizing it in a
spatial coordinate. Thus, we redefine the optimization problem with the
position of the payload along the $i\text{-axis}$ $x_{p}$ as the variable that
we discretize the dynamics along with. In other words, this means that we
perform a change of variable and now the system’s dynamics are described using
$dx/dx_{p}$.

###  III-A The variable change

A change of the integration variable is now performed and the time-optimal
problem is reformulated in the same fashion as in [12] and [13]. First,
consider the total time $T$ that the payload takes to go from the initial to
the final position i.e., the cost function in (7) is

$T=\int_{0}^{t_{f}}dt.$ (8)

Now, note that the first dynamic equation, used to describe the payload
position along the $i\text{-axis}$, in (6) is

$\dot{x}_{1}=\frac{dx_{1}}{dt}=x_{2}\implies\frac{dt}{dx_{1}}=\frac{1}{x_{2}}.$
(9)

In this way, the cost function to be minimized is rewritten as

$T=\int_{0}^{t_{f}}dt=\int_{x_{1_{0}}}^{x_{1_{f}}}\frac{dx_{1}}{x_{2}}=\int_{x_{p_{0}}}^{x_{p_{f}}}\frac{dx_{p}}{x_{2}}.$
(10)

The variable change implies that the optimization problem is now parametrized
in the payload position along the $i\text{-axis}$ ($x_{1}=x_{p}$) and consists
of minimizing the integral on the right-hand side of (10). The main benefit of
this will be discussed in the next section. Nevertheless, an immediate
consequence of (10) is that a new state vector
$x=[t,\dot{x}_{p},y_{p},\dot{y}_{p},l,\dot{l},\theta,\dot{\theta}]^{T}$ is
defined for the problem.

Remark: With the variable change, time $t$ is no longer the free variable.
Now, all the derivatives will be with respect to $x_{p}$ and thus, with
different notation, $x^{\prime}=dx/dx_{p}$.

Additionally, since time $t$ is now a state variable, we make the following
identification

$x_{1}\leftarrow{t},~{}x^{\prime}_{j}\leftarrow{\frac{dx_{j}}{dx_{p}}},~{}j=1,\ldots,n,$
(11)

and the state equations in (6) become a system of differential algebraic
equations

$\displaystyle x_{2}x^{\prime}_{1}$ $\displaystyle=1$ (12) $\displaystyle
x_{2}x^{\prime}_{2}$ $\displaystyle=-(u_{2}\sin(x_{7}))/m_{2}$ $\displaystyle
x_{2}x^{\prime}_{3}$ $\displaystyle=x_{4}$ $\displaystyle x_{2}x^{\prime}_{4}$
$\displaystyle=-(u_{2}\cos(x_{7}))/m_{2}+g$ $\displaystyle
x_{2}x^{\prime}_{5}$ $\displaystyle=x_{6}$ $\displaystyle x_{2}x^{\prime}_{6}$
$\displaystyle=\leavevmode\resizebox{399.2946pt}{}{$x_{5}x_{8}^{2}+g\cos(x_{7})-u_{2}/m_{2}-\sin(x_{7})(u_{1}+u_{2}\sin(x_{7}))/m_{1}$}$
$\displaystyle x_{2}x^{\prime}_{7}$ $\displaystyle=x_{8}$ $\displaystyle
x_{2}x^{\prime}_{8}$
$\displaystyle=\leavevmode\resizebox{399.2946pt}{}{$-(2x_{6}x_{8}+g\sin(x_{7})+\cos(x_{7})(u_{1}+u_{2}\sin(x_{7}))/m_{1})/x_{5},$}$

where $j$ indices the state variables in the new state vector and $n$ is the
system’s dimension. It is worth noting that in performing the coordinate
change, we implicitly make an assumption that the payload is moving
monotonically, in one direction along $i\text{-axis}$. This is a natural
limitation that implicitly implies a no-sway condition in the solution.

Subsequently to the change of variable made in (10), a natural choice of the
cost function $J$ would be

$J=\int_{x_{p_{0}}}^{x_{p_{f}}}\frac{1}{x_{2}(x_{p})}dx_{p},$ (13)

which leads to convergence issues since $x_{2}(x_{p})=0$ at $x_{2}(0)$ and
$x_{2}(x_{2_{f}})$. However, an interesting property of the reformulation is
that the variable $x_{1}$ in (12) corresponds to time $x_{1}(x_{p})=t$. Thus,
the optimization problem can now be written as

$\displaystyle\underset{}{\text{minimize}}$ $\displaystyle J=t(x_{p_{f}})$
(14) subject to $\displaystyle
x_{2}x^{\prime}(x_{p})=f(x_{p},x(x_{p}),u(x_{p}))$ $\displaystyle x(0)=x_{0}$
$\displaystyle x(x_{p_{f}})=x_{f}$ $\displaystyle 0\leq t(x_{p})$
$\displaystyle 0\leq x_{2}(x_{p})$ $\displaystyle g(x_{p})\leq x(x_{p})\leq
f(x_{p})$ $\displaystyle u_{min}(x_{p})\leq u(x_{p})\leq u_{max}(x_{p}).$

As mentioned before and more evident now, the positions, velocities and
accelerations are related to one another through the parametrization of path.
This is an interesting feature for representing the geometric constraints as
we will see in the next section. Furthermore, though the general dynamics
remain nonlinear and non-convex, the cost function is nevertheless still
convex, which is an additional outcome of the reformulation.

##  IV GEOMETRIC CONSTRAINTS

When dealing with the constraints that the different heights of the stacks
impose on the payload height $y_{p}(x_{p})$, a function $s(x_{p})$ that
represents the stack profile along the loading site is implicitly required.
This way, the constraints on $y_{p}(x_{p})$ are

$0\leq y_{p}(x_{p})\leq h-s(x_{p}),$ (15)

where $h$ is the maximum height, e.g., the distance from the ground to the
trolley.

With the optimization problem in the original formulation as in (7), after the
time discretization, the constraints imposed on $y_{p}(x_{p})$ in (15) would
be

$0\leq y_{p}(x_{p}(t^{k}))\leq h-s(x_{p}(t^{k})).$ (16)

Moreover, the required explicit function representation $s(x_{p}(t^{k}))$ will
generally be discontinuous, nonlinear and non-convex.

However, with the optimization problem (14) now parametrized in $x_{p}$, the
geometric constraints can be easily represented. The height of the stacks and
their positions along the trajectory are represented by the function
$s(x_{p})$, which now addresses the stack height at each position $x_{p}$.
This way, when discretizing $x_{p}$, the function $s(x_{p})$ determines the
upper bound constraints for $y_{p}(x_{p})$ in (14) as

$0\leq y_{p}(x_{p}^{k})\leq h-s(x_{p}^{k}).$ (17)

Note that we no longer need an explicit function $s(x_{p})$, but simply
function values that can be computed when setting up the numerical model. See
Fig. 3.

Figure 3: Two-dimensional schematic representing the trolley and payload
motions and the geometric constraints (stacks).

This way of representing the stack heights and their positions is attractive
since it is easier and more intuitive to be constructed and manipulated, after
the discretization. This is the main benefit of the variable change in
(9)-(10) and constitutes the main contribution of this paper.

##  V RESULTS AND DISCUSSION

In order to illustrate and validate the idea, a scenario of stack
configuration was simulated. For simplicity, a small-scale example was used,
where the integration goes from position $x_{p_{0}}=0$ to $x_{p_{f}}=1$. In
this scenario, the container stacks are particularly high at the end of the
loading site.

The software used to obtain the results was MATLAB. Additionally, CasADi
software tool [14] and Yop toolbox [15] were used for modeling the
optimization problem, and IPOPT [16] was used to solve it.

Now, it is important to note that all the distances and lengths used in the
results are in meters and the angles in radian. With (14) in mind, the initial
and final conditions were set to

$\displaystyle t(0)=0,~{}$ (18) $\displaystyle x_{p}(0)=0,$ $\displaystyle
x_{p}(x_{p_{f}})=1,$ $\displaystyle\dot{x}_{p}(0)=0,~{}$
$\displaystyle\dot{x}_{p}(x_{p_{f}})=0,$ $\displaystyle y_{p}(0)=3,~{}$
$\displaystyle y_{p}(x_{p_{f}})=3,$ $\displaystyle\dot{y}_{p}(0)=0,~{}$
$\displaystyle\dot{y}_{p}(x_{p_{f}})=0,$ $\displaystyle l(0)=3,~{}$
$\displaystyle l(x_{p_{f}})=3,$ $\displaystyle\dot{l}(0)=0,~{}$
$\displaystyle\dot{l}(x_{p_{f}})=0,$ $\displaystyle\theta(0)=0,~{}$
$\displaystyle\theta(x_{p_{f}})=0,$ $\displaystyle\dot{\theta}(0)=0,~{}$
$\displaystyle\dot{\theta}(x_{p_{f}})=0,$

the constraints regarding the minimum and maximum height that the payload can
be, given its position $x_{p}$ along the $i\text{-axis}$, were defined as

$0.15\leq y_{p}(x_{p})\leq h-s(x_{p}),$ (19)

and the other box constraints were

$\displaystyle 0\leq$ $\displaystyle t(x_{p})$ (20) $\displaystyle 0\leq$
$\displaystyle\dot{x}_{p}(x_{p})$ $\displaystyle 0\leq$ $\displaystyle
l(x_{p})\leq 4.5$ $\displaystyle-0.1\leq$ $\displaystyle\theta(x_{p})\leq 0.1$
$\displaystyle-1\leq$ $\displaystyle F_{t}(x_{p})\leq 1$ $\displaystyle 0\leq$
$\displaystyle F_{h}(x_{p})\leq 8.$

Additionally, a first-order polynomial degree was used for the collocation
method in the solver, the problem was solved in $100$ control intervals, and
the trolley and payload masses were respectively set to $m_{1}=1.2kg$ and
$m_{2}=0.6kg$.

Finally, the position of each stack was assigned as in (21), where
$c\in\mathbb{R}^{m}$ represents the positions where the stacks are along the
$i\text{-axis}$. In this example, $9$ stack positions were assigned, separated
by $10cm$ from each other.

$c=\begin{bmatrix}0.1&0.2&0.3&0.4&0.5&0.6&0.7&0.8&0.9\end{bmatrix}.$ (21)

Moreover, the height of each stack was represented as in (22)

$\displaystyle\rho=\begin{bmatrix}0.5&1.0&1.0&1.0&2.0&2.0&2.4&2.5&1.0\end{bmatrix},$
(22)

where $\rho\in\mathbb{R}^{m}$ represents the height of each of the stacks,
corresponding to the assigned position in (21). Also, the width of the stacks
was considered, here $0.08m$. From these, the function values in the bound
constraints (17) can be constructed.

###  V-A Results

Fig. 4 shows the path followed by the payload, given the stack heights (22) at
the positions (21).

Figure 4: The path followed by the payload subject to stack constraints.

Figures 5 and 6 show how the payload positions, hoisting and sway evolve with
time. Note that though they correspond to the decision variables in (6), they
are the result of problem solved as in (14).

Figure 5: The payload spatial-coordinates trajectories. Figure 6: The payload
hoisting and sway trajectories.

Lastly, Fig. 7 shows the velocity $\dot{x}_{p}$, time (which now is a decision
variable), and control inputs $F_{t}$ and $F_{h}$ evolving along $x_{p}$.

Figure 7: How $\dot{x}_{p}$, time and control inputs evolve with the spatial
coordinate.

###  V-B Discussion

The presented numerical example shows the simulation results for the
optimization control problem, solved off-line after a change of variable. The
aim was to illustrate that with the variable change, the description of the
stack heights becomes trivial and easier to represent numerically. This is
done by defining their positions and corresponding heights in $c$ and $\rho$,
respectively. These are then implicitly defining a function $s(x_{p})$, which
addresses the stacks’ height in each discretization interval. Moreover, Figs.
4 to 7 show that the payload was transferred from the initial to the final
position avoiding the stacks and not violating the predefined constraints.
Hence, we believe that the results have further strengthened our idea that
this problem becomes trivial with the discretization of $x_{p}$.

We are aware that our work still has some limitations. As mentioned before, a
natural one is that the solution enforces no sway condition since the payload
is moving monotonically in one direction. Furthermore, the distance traveled
by the payload in each discretization interval varies along the loading and
unloading process. For instance, as seen in Fig. 4, they were particularly
long in the end, after a tall stack. Thus, since the problem is uniformly
discretized, most of the dynamics that are not captured might be in those
parts. This can be easily addressed by non-uniform discretization. Notice that
these regions, where more time is spent, are strongly influenced by the
stacks’ disposition.

Perhaps the biggest limitation of this work is that, though the objective
function and the constraints on the container heights are now convex, the
dynamics of the system remain non-convex. This might lead to a solution at a
local minimum. However, it is important to note that circumventing these
limitations is not in the scope of this work. Moreover, non-convexity is
simply a consequence of the high-fidelity model of the system. Additionally,
the problem may be extended to 3D, however, we believe that this is not
necessary, given the nature of ship-to-shore crane operations.

##  VI CONCLUSIONS AND FUTURE WORK

We have proposed a new approach to deal with the different heights that the
container stacks can assume during the process of loading and unloading
container ships. With this in mind, a variable change has been performed and
the representation of the stack heights became trivial in the optimization
problem. Subsequently, a small-scale example has led us to confirm it. Thus,
this approach has the potential to be successfully applied to real crane
operations and enhance their automation.

Future work will focus on going beyond the point-mass assumption and
incorporate more physical and geometric constraints to the setup. Moreover, we
will investigate how to deal with it when the initial or final position is in
between stacks. Additionally, different densities of discretization will be
addressed.

## References

  * [1] E. Kreuzer, M.-A. Pick, C. Rapp, and J. Theis, “Unscented kalman filter for real-time load swing estimation of container cranes using rope forces,” _Journal of Dynamic Systems, Measurement, and Control_ , vol. 136, no. 4, Apr. 2014. [Online]. Available: https://doi.org/10.1115/1.4026602
  * [2] Y. Sakawa and Y. Shindo, “Optimal control of container cranes,” _Automatica_ , vol. 18, no. 3, pp. 257–266, May 1982. [Online]. Available: https://doi.org/10.1016/0005-1098(82)90086-3
  * [3] A. Arena, A. Casalotti, W. Lacarbonara, and M. Cartmell, “Dynamics of container cranes: three-dimensional modeling, full-scale experiments, and identification,” _International Journal of Mechanical Sciences_ , vol. 93, pp. 8–21, Apr. 2015. [Online]. Available: https://doi.org/10.1016/j.ijmecsci.2014.11.024
  * [4] Y.-S. Kim, K.-S. Hong, and S.-K. Sul, “Anti-sway control of container cranes: inclinometer, observer, and state feedback,” _International Journal of Control, Automation, and Systems_ , vol. 2, no. 4, pp. 435–449, 2004.
  * [5] M. J. Maghsoudi, Z. Mohamed, A. Husain, and M. Tokhi, “An optimal performance control scheme for a 3d crane,” _Mechanical Systems and Signal Processing_ , vol. 66-67, pp. 756–768, Jan. 2016. [Online]. Available: https://doi.org/10.1016/j.ymssp.2015.05.020
  * [6] J. Auernig and H. Troger, “Time optimal control of overhead cranes with hoisting of the load,” _Automatica_ , vol. 23, no. 4, pp. 437–447, Jul. 1987\. [Online]. Available: https://doi.org/10.1016/0005-1098(87)90073-2
  * [7] H. Chen, Y. Fang, and N. Sun, “Optimal trajectory planning and tracking control method for overhead cranes,” _IET Control Theory & Applications_, vol. 10, no. 6, pp. 692–699, Apr. 2016. [Online]. Available: https://doi.org/10.1049/iet-cta.2015.0809
  * [8] A. Al-Garni, K. Moustafa, and S. J. Nizami, “Optimal control of overhead cranes,” _Control Engineering Practice_ , vol. 3, no. 9, pp. 1277–1284, Sep. 1995. [Online]. Available: https://doi.org/10.1016/0967-0661(95)00126-f
  * [9] J. D. Cruz and F. Leonardi, “Minimum-time anti-swing motion planning of cranes using linear programming,” _Optimal Control Applications and Methods_ , vol. 34, no. 2, pp. 191–201, Feb. 2012. [Online]. Available: https://doi.org/10.1002/oca.2016
  * [10] K.-S. Hong and U. H. Shah, _Lumped Mass Models of Gantry Cranes_. Singapore: Springer Singapore, 2019, pp. 11–34. [Online]. Available: https://doi.org/10.1007/978-981-13-5770-1_2
  * [11] C. Rosmann, F. Hoffmann, and T. Bertram, “Timed-elastic-bands for time-optimal point-to-point nonlinear model predictive control,” in _2015 European Control Conference (ECC)_. IEEE, Jul. 2015. [Online]. Available: https://doi.org/10.1109/ecc.2015.7331052
  * [12] D. Verscheure, B. Demeulenaere, J. Swevers, J. D. Schutter, and M. Diehl, “Time-optimal path tracking for robots: A convex optimization approach,” _IEEE Transactions on Automatic Control_ , vol. 54, no. 10, pp. 2318–2327, Oct. 2009. [Online]. Available: https://doi.org/10.1109/tac.2009.2028959
  * [13] K. Shin and N. McKay, “Minimum-time control of robotic manipulators with geometric path constraints,” _IEEE Transactions on Automatic Control_ , vol. 30, no. 6, pp. 531–541, Jun. 1985. [Online]. Available: https://doi.org/10.1109/tac.1985.1104009
  * [14] J. A. E. Andersson, J. Gillis, G. Horn, J. B. Rawlings, and M. Diehl, “CasADi: a software framework for nonlinear optimization and optimal control,” _Mathematical Programming Computation_ , vol. 11, no. 1, pp. 1–36, Jul. 2019. [Online]. Available: https://doi.org/10.1007/s12532-018-0139-4
  * [15] V. Leek, “An optimal control toolbox for matlab based on casadi,” 2016.
  * [16] A. Wächter and L. T. Biegler, “On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming,” _Mathematical Programming_ , vol. 106, no. 1, pp. 25–57, Apr. 2005. [Online]. Available: https://doi.org/10.1007/s10107-004-0559-y


$\nabla\ell_{\tau},\nabla\ell^{\tau},\nabla\ell\vbox{\hbox{$\scriptstyle\tau$}}$ | LPPM updates (17)
$\Delta\ell_{\tau},\Delta\ell^{\tau},\Delta\ell\vbox{\hbox{$\scriptstyle\tau$}}$ | LPPM finite-differences (41)
$\widetilde{\ell}$ | linearization of $\ell$ at $x^{*}$ (6)
$\widetilde{\ell}_{\tau},\widetilde{\ell}^{\tau},\widetilde{\ell}\vbox{\hbox{$\scriptstyle\tau$}}$ |  $\mathcal{L}$-envelopes of lin. loss $\widetilde{\ell}$
$\widetilde{z}_{\tau}=(\widetilde{x}_{\tau},\widetilde{y}_{\tau})$ | lin. lower $\mathcal{L}$-proximal map (22)
$\widetilde{z}^{\tau}=(\widetilde{x}^{\tau},\widetilde{y}^{\tau})$ | lin. upper $\mathcal{L}$-proximal map (23)
$\nabla\widetilde{\ell}_{\tau},\nabla\widetilde{\ell}^{\tau},\nabla\widetilde{\ell}\vbox{\hbox{$\scriptstyle\tau$}}$ | LPGD updates (24)
$\Delta\widetilde{\ell}_{\tau},\Delta\widetilde{\ell}^{\tau},\Delta\widetilde{\ell}\vbox{\hbox{$\scriptstyle\tau$}}$ | LPGD finite-differences
$\rho$ | augmentation strength parameter
$\mathcal{L}_{\rho}$ | augmented Lagrangian (28)
$\ell_{\tau\rho},\ell^{\tau\rho},\ell\vbox{\hbox{$\scriptstyle\tau\rho$}}$ | aug. $\mathcal{L}$-envelopes (29)
$z_{\tau\rho}=(x_{\tau\rho},y_{\tau\rho})$ | aug. lower $\mathcal{L}$-proximal map (30)
$z^{\tau\rho}=(x^{\tau\rho},y^{\tau\rho})$ | aug. upper $\mathcal{L}$-proximal map
$\nabla\ell_{\tau\rho},\nabla\ell^{\tau\rho},\nabla\ell\vbox{\hbox{$\scriptstyle\tau\rho$}}$ | aug. LPPM updates (31)
$\Delta\ell_{\tau\rho},\Delta\ell^{\tau\rho},\Delta\ell\vbox{\hbox{$\scriptstyle\tau\rho$}}$ | aug. LPPM finite-differences (42)
$\widetilde{\ell}_{\tau\rho},\widetilde{\ell}^{\tau\rho},\widetilde{\ell}\vbox{\hbox{$\scriptstyle\tau\rho$}}$ | aug. $\mathcal{L}$-envelopes of lin. loss $\widetilde{\ell}$
$\widetilde{z}_{\tau\rho}=(\widetilde{x}_{\tau\rho},\widetilde{y}_{\tau\rho})$ | aug. lin. lower $\mathcal{L}$-proximal map
$\widetilde{z}^{\tau\rho}=(\widetilde{x}^{\tau\rho},\widetilde{y}^{\tau\rho})$ | aug. lin. upper $\mathcal{L}$-proximal map
$\nabla\widetilde{\ell}_{\tau\rho},\nabla\widetilde{\ell}^{\tau\rho},\nabla\widetilde{\ell}\vbox{\hbox{$\scriptstyle\tau\rho$}}$ | aug. lin. LPGD updates (31)
$\Delta\widetilde{\ell}_{\tau\rho},\Delta\widetilde{\ell}^{\tau\rho},\Delta\widetilde{\ell}\vbox{\hbox{$\scriptstyle\tau\rho$}}$ | aug. lin. LPGD
| finite-differences (42)
$L$ | loss on primal & dual variables
$\widetilde{L}$ | linearization of $L$ at $z^{*}$


Copyright for this paper by its authors. Use permitted under Creative Commons
License Attribution 4.0 International (CC BY 4.0).

5th International GamiFIN Conference 2021 (GamiFIN 2021), April 7-10, 2021,
Finland

[orcid=0000-0003-3928-6520<EMAIL_ADDRESS>]

[orcid=0000-0002-0388-3115<EMAIL_ADDRESS>]

[orcid=0000-0003-0343-3701<EMAIL_ADDRESS>]

[orcid=0000-0003-2681-8698<EMAIL_ADDRESS>]

[orcid=0000-0002-9730-2253<EMAIL_ADDRESS>]

[orcid=0000-0002-6573-588X<EMAIL_ADDRESS>]

[orcid=0000-0003-1574-0784<EMAIL_ADDRESS>]

# Does gamification affect flow experience? A systematic literature review

Wilk Oliveira  Institute of Mathematics and Computer Science, University of
São Paulo, São Carlos, São Paulo, Brazil  Department of Information Systems,
Brno University of Technology, Brno, Czech Republic  Gamification Group,
Faculty of Information Technology and Communication Sciences, Tampere
University, Tampere, Finland  Olena Pastushenko  Luiz Rodrigues  Armando M.
Toda  Paula T. Palomino  Juho Hamari  Seiji Isotani

(2020)

###### Abstract

In recent years, studies in different areas have used gamification to improve
users’ flow experience. However, due to the high variety of the conducted
studies and the lack of secondary studies (e.g., systematic literature
reviews) in this field, it is difficult to get the state-of-the-art of this
research domain. To address this problem, we conducted a systematic literature
review to identify i) which gamification design methods have been used in the
studies about gamification and Flow Theory, ii) which gamification elements
have been used in these studies, iii) which methods have been used to evaluate
the users’ flow experience in gamified settings, and iv) how gamification
affects users’ flow experience. The main results show that there is growing
interest to this field, as the number of publications is increasing. The most
significant interest is in the area of gamification in education. However,
there is no unanimity regarding the preferred method of the study or the
effects of gamification on users’ experience. Our results highlight the
importance of conducting new experimental studies investigating how
gamification affects the users’ flow experience in different gamified
settings, applications and domains.

###### keywords:

Gamification Flow Theory User experience Gamified settings Literature review

##  1 Introduction

Gamification is “the use of game elements (such as levels, points, badges, and
others) in non-game contexts” [1] and its techniques are used to enhance users
experience in different areas, such as education [2], marketing [3],
healthcare [4], and others. Different studies conducted over the past few
years [5, 6, 7] have shown that, if correctly applied, gamification can affect
different types of positive psychological experiences (e.g., concentration,
learning, and flow) in the users [8, 6, 9].

One of the main characteristics of the users experience that might be
influenced by gamification is flow experience [7], defined as a “mental state
in which a person performing some activity is fully immersed in a feeling of
energized focus, full involvement, and enjoyment in the process of the
activity” [10, 11, 12]. This is because one of the goals of gamification is to
take users to a positive experience, some times, very related to flow [1].
Therefore, in recent years, different studies have proposed use various
gamified tools to keep users in the state of flow during some activity, as
well as, analyzing the effectiveness of different strategies to keep people in
a flow experience [13, 14, 15].

However, the results related to the effects of gamification on people’s flow
experience are still uncharted, and it is not possible to have an overview of
the state-of-the-art. To face this challenge, in this study we conducted a
systematic literature review to determine whether gamification techniques
positively affect the users’ flow state. To achieve this goal, we analyzed i)
what methods, tools and gamification elements have been mainly used in the
published studies, ii) what methods, techniques and tools have been used to
analyze the flow experience in these studies, and iii) what are the outcomes
of the studies of flow experience in gamified settings.

The literature review results show that i) there is an increasing interest to
studying the effects of gamification on flow experience, ii) education is the
main domain addressed in the studies, iii) there is no consensus on which
methods and gamification elements to use to bring study participants to a flow
experience, iv) the questionnaire is the only method used to analyze the
participant flow experience, and v) the the effects of gamification on the
flow experience are mixed (i.e., positive, negative, and neutral). Discussions
section (Section 3.1) reveals why more research in these areas is needed,
based on the analysis of the existing studies.

Based on this premise, the main contribution of this paper is that it
identified the most prevalent areas in the studies, which methods, techniques
and game elements used in the studies, which methods are used to identify the
users’ flow experience and which are the possible effects of the gamification
of the people’s flow experience. Thus, as far as we know, this is the first
study to provide an overview of the state-of-the-art in the field of
gamification and Flow Theory (i.e., flow experience). The remaining of this
paper is organized as follows. Section 2 presents the study method, Section 3
presents the results, discussion and limitations of the study, and Section 4
presents the study conclusion.

##  2 Method

This systematic literature review aimed to identify the state-of-the-art of
research on the effects of gamification on users’ flow experience. To achieve
this goal, we followed the well-known protocol proposed by Kitchenham [16]. It
defines three general steps (planning, conducting, and documenting) to conduct
secondary studies. Following this protocol, as the first step, we defined four
research questions (RQ):

  * •

RQ 1: What gamification design methods have been used in the studies about
gamification and Flow Theory?

  * •

RQ 2: What gamification elements have been used in these studies?

  * •

RQ 3: What methods have been used to evaluate the users’ flow experience in
gamified settings?

  * •

RQ 4: How gamification affects users’ flow experience?

Through the RQ 1 and RQ 2, it will be possible to have an overview of the main
methods, tools, and gamification elements used in the studies that relate
gamification and Flow Theory, allowing then to identify, for example, if there
is a consensus concerning use a specific group of gamification elements, or if
there is space for the use of new methods, tools or gamification elements in
future studies in this field. Through RQ 3, it will be possible to identify
how the participants’ flow experience is analyzed in gamified settings, and
consequently, as if there is a consensus about using a specific strategy to
analyze the participants’ flow experience. Finally, through RQ 4, it will be
possible to get a general sense of the results of using gamification in the
participants’ flow experience. We believe that this set of questions will
allow us to have a general notion of the state-of-the-art in studies that
relate gamification and Flow Theory.

The next step of finding the primary studies, based on the selected protocol,
is defining a “search string”. Following different recent literature reviews
[6, 17], we defined the search string based on the PICOC (Population,
Intervention, Comparison, Outcomes, and Context) method described by
Kitchenham and Charters [18]. Thus, the following PICOC was defined:

  * •

Population: studies that describe, apply or evaluate gamification to provide
users’ flow experience;

  * •

Intervention: methods used to provide or evaluate users’ flow experience in
gamified systems;

  * •

Comparison: not applicable, since the purpose of this study is to describe the
state-of-the-art;

  * •

Outcomes: most used methods to provide or evaluate users’ flow experience in
gamified systems;

  * •

Context: studies in the field of gamification.

After applying the PICOC method, the generated search string was validated by
a comparison with topics presented in recent studies in this field [19, 6, 9,
17]. We also validated our search string with two experts in the areas of
gamification and Flow Theory. Thus, the final search string is: “gamification
AND (flow theory OR flow experience OR flow state)”.

The next step was to define the sources. We conducted this step based on other
recent literature reviews in the field of gamification and flow experience [6,
9, 17, 19]. Thus we defined four sources: ACM Digital
Library111https://dl.acm.org/; IEEE Xplorer222https://ieeexplore.ieee.org/;
Science Direct333https://www.sciencedirect.com/; Springer
Link444http://link.springer.com/. In the next step, we defined inclusion (IC)
and exclusion criteria (EC):

  * •

IC1: primary studies about gamification and flow theory;

  * •

EC1: secondary and tertiary studies;

  * •

EC2: redundant studies (written by the same authors and addressing the same
topic555For these cases, we used the most recent studies)

  * •

EC3: gray literature (non peer-reviewed studies).

Afterwards, we defined the data to be extracted from each selected study:

  1. 1.

study information (reference, title, authors list, authors’ country, authors’
affiliations, source type (journal or conference), source, publishing year,
and abstract (based on Oliveira et al. [19]));

  2. 2.

application domain;

  3. 3.

method used to provide users’ flow experience;

  4. 4.

used gamification elements;

  5. 5.

method used to identify users’ flow experience;

  6. 6.

Study results (outcomes).

The data collection process was conducted in October (2020) and was conducted
by two experts in gamification and Flow Theory. They are also experts in
conducting secondary studies. The experts studied the title, abstract and
general metadata of all studies. Inspired by recent secondary studies [17,
19], doubts regarding the inclusion of studies (i.e., whether or not a study
is relevant to our research questions) were discussed between the two
researchers. A web system for the management of secondary studies
(Parsif.al666https://parsif.al/) was used to ease the process of the
literature review (e.g., managing which studies were already analyzed, whether
the studies were rejected, performing quality assessment and data extraction).

##  3 Results

This section presents the literature review results, starting with the
demographic information, and then answering the research questions afterward.
In the first phase of the study, after executing the search string in the four
research sources, 411 studies were found (ACM Digital Library = 71; IEEE
Digital Library = 14; Science Direct = 95; and Springer Link = 231). Then,
after removing the duplicate studies, 408 studies remained. After reading the
studies’ title and abstract, we found 25 studies, and finally, after the
complete reading, we found 19 studies (ACM Digital Library = 3; IEEE Digital
Library = 1; Science Direct = 4; and Springer Link = 11) that answered at
least one of the selected research questions (and were corresponding to the
inclusion criteria). Figure 1 presents an overview for the filtering process
and Table 1 presents the list of included studies. To facilitate the studies
references throughout the text, we create ‘Ids’ in the Table 1, which will be
used whenever we are referring to a study.

Figure 1: Filtering Table 1: List of selected studies Id |  Title | Ref.
---|---|---
S01 |  A Methodology for Multimodal Learning Analytics and Flow Experience Identification within Gamified Assignments | [20]
S02 |  How a company’s gamification strategy influences corporate learning: A study based on gamified MSLP (Mobile social learning platform) | [21]
S03 |  A Novel Approach to Interactive Dialogue Generation Based on Natural Language Creation with Context-Free Grammars and Sentiment Analysis | [22]
S04 |  Psychological effects of gamified didactics with exergames in Physical Education at primary schools: Results from a natural experiment | [23]
S05 |  What motivates visitors to participate in a gamified trip? A player typology using Q methodology | [24]
S06 |  Understanding the Effect of Gamification of Learning Using Flow Theory | [25]
S07 |  Gamification in Local Intangible Cultural Heritage Museums for Children: A Case Design | [26]
S08 |  Gameful Experience Questionnaire (GAMEFULQUEST): an instrument for measuring the perceived gamefulness of system use | [27]
S09 |  Learning for a Healthier Lifestyle Through Gamification: A Case Study of Fitness Tracker Applications | [28]
S10 |  gMOOCs – Flow and Persuasion to Gamify MOOCs | [29]
S11 |  Improving Video Engagement by Gamification: A Proposed Design of MOOC Videos | [30]
S12 |  Strategies for Playful Design When Gamifying Rehabilitation: A Study on User Experience | [31]
S13 |  Gamification Methods in Higher Education | [32]
S14 |  Goal-Based Manufacturing Gamification: Bolt Tightening Work Redesign in the Automotive Assembly Line | [33]
S15 |  Contextual Gamification of Social Interaction – Towards Increasing Motivation in Social E-learning | [34]
S16 |  Measuring flow in gamification: Dispositional Flow Scale-2 | [35]
S17 |  Competition as an Element of Gamification for Learning: An Exploratory Longitudinal Investigation | [36]
S18 |  Combining Augmented Cognition and Gamification | [37]
S19 |  Context-Sensitive User-Centered Scalability: An Introduction Focusing on Exergames and Assistive Systems in Work Contexts | [38]

The first study was published in 2012, and the majority of the studies were
published in 2020. Figure 2 shows the number of publications over the years.
Overall, the results indicate that there is a growing interest in researching
the topic. In total, 64 different researchers, from 14 countries (Germany
(five studies), China (three studies), Republic of Korea, Finland, USA, Canada
(two studies each), Czech republic, Sweden, Spain, Netherlands, Brazil, United
Kingdom, Bosnia and Herzegovina, Portugal (one study each)) contributed to 19
selected studies, and only one author participated in more than one study.
Figure 3 presents heat-map of publications according to the number of
publications.

Figure 2: Timeline of published studies Figure 3: Publications map

From the 19 selected studies, five are exploratory studies (S02, S03, S05, S13
and S17), five are experimental studies (S01, S04, S06, S07 and S14), four are
proposals (S10, S11, S18 and S19), two are case studies (S09 and S15), two are
psychometric studies (S08 and S16), and one is a quasi-experimental study
(S12). Eleven from the 19 selected studies were conducted in the field of
education (S01, S02, S03, S04, S06, S10, S11, S13, S15, S17, S18), two in the
field of health (S09, S12), two in the field of tourism (S05, S07), two in the
field of general gamification (S08, S16), one the field of industry (S14) and
one in the field of usability (S19). Figure 4 presents a relation between the
domains and type of studies.

Figure 4: Type of studies and domain

RQ 1: What gamification design methods have been used in the studies about
gamification and Flow Theory?. Only four from the 19 selected studies did not
present the method used to provide the flow experience (S02, S05, S08, S16).
The other 15 studies presented the used method. However, only two (S01, S12)
also presented the used tools. S01 and S18 used gamified assignments (the
gamified assignments used by S01 is called bombsQuery), S03 used a gamified
dialogue scene, S04 used gamified exergaming, S06 used a gamified educational
system, S07 used a gamified mock-up, S09 used a gamified fitness tracking
application, S10 and S11 used gamified MOOCS, S12 used a gamified
rehabilitation system (called HUMAC NORM Testing), S13 used a gamified
classrooms (sometimes using tailored gamification resources), S14 used a
gamified bolt tightening tool, S15 used a gamified social e-learning
environment, S17 used a gamified simulation app, and S19 used a gamified
assistive systems.

RQ 2: What gamification elements have been used in these studies?. Seven from
19 studies did not specify the gamification elements used in the study (S01,
S02, S03, S05, S08, S12 ans S16). Based on the studies that reported the
gamification elements used, we identified that 31 different elements were used
in the studies. The gamification elements “Level” and “Points” were the most
used, being used in five different studies, followed by the gamification
elements “Badges” and “Leaderboards”, used in four different studies. The
gamification element “Goal” was used in three different studies. Table 2
presents an overview on the used gamification elements throughout the selected
studies, while Table 3 organizes the methods, tools, and gamification elements
used in each study.

Table 2: Used gamification elements Gamification elements | Count
---|---
Level, Points | 5
Badges, Leaderboards | 4
Goals | 3
Countdown, Rewards, Feedback, Immediate feedback, Progress bar, Storytelling | 2
Achievements, Audio-visual feedback, Audio-visual feedback, Avatars, Challenges, Clues, Collaboration, Communication Channels, Community Features, Competition, Customization, Documentation, Empowerment, Flexibility, Guilds, Minigames, Narrative, Quests, Skill trees, Smooth Learning Curves, Starts | 1
Table 3: Design methods, tools, and gamification elements Study | Design method | Tools |  Gamification elements
---|---|---|---
S01 | Gamified assignment | bombsQuery |  -
S03 | Dialogue scene | - |  -
S04 | Gamified exergaming | - |  Points, Badges and Leaderboards
S06 | Gamified educational system | - |  Immediate Feedback, Rewards and Points
S07 | Gamified mock-up | - |  Level, Starts and Feedback
S09 | Fitness tracking application | - |  Goals, Points, Levels, Progress Bars, Feedback, Documentation, Badges, Leaderboards, Time, Quests, Avatars, Storytelling and Community Features
S10 | MOOC | - |  Empowerment, Smooth Learning Curves, Communication Channels, Levels, Clues, Goal Indicators, Skill trees, Guilds, Storytelling, Points, Badges and Leaderboards
S11 | Videos MOOC | - |  Narrative
S12 | Rehabilitation System | HUMAC NORM Testing |  -
S13 | Gamified classrooms | - |  Points, Challenges, Collaboration, Minigames, Achievements, Leaderboard, Levels, Extra rewards, Flexibility and Countdown
S14 | Bolt tightening tool | - |  Audio-visual feedback, Progress bar and badges
S15 | Social e-learning environment | - |  Goals, Immediate feedback and Customization
S17 | Simulation app | - |  Competition
S18 | Gamified Activity | - |  -
S19 | Assistive systems | - |  Level

RQ 3: What methods have been used to evaluate the users’ flow experience in
gamified settings?. Eleven from the 19 selected studies illustrated some
method for analyzing the flow experience (the following studies did not
specify a method used to analyze the flow experience: S09, S10, S11, S12, S13,
S15, S18, S19). Coincidentally, all studies that reported the method used,
used the same method to analyze the students’ flow experience (i.e.,
questionnaire). However, different questionnaires were used (see Table 4).
Additionally, S08 conducted a Psychometric study to validate a questionnaire
to identify the GAMEFUL experience [39] (GAMEFULQUEST), which includes the
flow experience with one of its dimensions. S16 conducted a Psychometric study
to validate the flow state scale proposed and validated by Jackson and Eklund
[40] for the gamification domain.

Table 4: Flow experience evaluation tools Studies |  Tools
---|---
S01, S07 |  Short Flow State Scale-2 [40]
S02 |  Four questions adjusted by Agarwal and Karahanna [41]
S03 |  Game experience questionnaire (GEQ) [42]
S04 |  Dispositional Flow State Scale-2 [40]
S05 |  Q sorts questionnaire adapted from Van Exel and De Graaf [43]
S06 |  Flow questionnaire Csikszentmihalyi and Larson [11]
S14 |  Experience sampling method (ESM) [44]
S17 |  Multidimensional measurement Mahfouz and Guo [45]

RQ 4: How gamification affects users’ flow experience?. Ten from the 19
selected studies present some outcome about the effects of gamification on the
users’ experience (in the following studies, authors did not report the
results: S07, S09, S10, S11, S12, S13, S15, S18, S19). Overall, the results
are mixed. On the one hand, the results of S03, S04, S05, S06, and S17
indicated that gamification did not significantly affect the users’ flow
experience. On the other hand, S01 identified that gamification positively
affected the users’ flow experience (in general, the users’ flow experience in
the gamified settings was high = 4.4). S02 demonstrated that Challenge
(gamification design proposed to encourage challenge among participants),
Relationship (gamification design proposed to encourage relationship among
participants), and Usability positively affected flow and flow affected the
intention of continuous use. S14 identified that a gamified interface improved
worker’s flow experience compared to other interfaces. Finally, S08 and S16
obtained good model fits to validate its scales (i.e., GAMEFUL questionnaire
[27] and FSS2 [35]). Table 5 presents a comparison analyzing the types of
outcomes in terms of domain, type of study and gamification elements.

Table 5: Outcomes comparison Weak effect | Strong positive effect
---|---
Domain |  Type of study |  Gamification elements |  Domain |  Type of study |  Gamification elements
Education (S03; S04; S06; S17) Tourism (S05) |  Exploratory (S03; S05; S17) Experiment (S04; S06) |  Points (S04, S06), Badges (S04), Leaderboards (S04), Immediate feedback (S06), Rewards (S06), Competition (S17) |  Education (S01; S02), Industry (S14) |  Experiment (S01; S14), Exploratory (S02) |  Audio-visual feedback, Progress bar, Badges (S14)

In summary, the results indicate that i) there is a growing interest of the
community in analyzing the effects of gamification on the users’ flow
experience; ii) the types of studies are varied, but most studies are
exploratory and quantitative; iii) there is a predominance of research in the
field of education; iv) many gamified methods have been used to provide the
flow experience, but there is no unanimity in the type of method (although
most can be generally characterized as gamified systems); v) several
gamification elements have been used, however, Level, Points, Badges, and
Leaderboards are predominant; vi) a single method has been used to analyze the
users’ flow experience (i.e., questionnaires), but there is no consensus
concerning which questionnaire to use; vii) there is no unanimity regarding
gamification’s effect on the users’ flow experience, although promising
results are observed.

###  3.1 Discussions

In this paper, we presented the results of a secondary study on the field of
gamification and Flow Theory. The overall results of our systematic literature
review showed that most studies were conducted in the area of education. This
result corresponds with the results observed in different secondary studies
carried out in the large area of gamification [8, 6, 9] who also identified
that education is the area with the largest number of studies on gamification.
This observation highlights the importance of conducting research analyzing
how gamification affects the flow experience of users in specific contexts
where there are few to no studies (e.g., marketing, tourism, health).

The studies on gamification started to become popular in the year 2012 [1, 8,
35]. It was precisely in that year that the first study on the use of
gamification in the one related to Flow Theory appeared (S19). This discovery
indicates that since the first studies on gamification, it was already thought
of as a mechanism capable of influencing the people’s flow experience.
However, the number of studies in this research domain started to increase
from 2016. This result shows that there is a growing interest of researchers
to investigate the effects of gamification on people’s flow experience, as
well as that there is a growth perspective in the number of studies in this
field (see Figure 2).

At the same time, most of the published studies so far are exploratory studies
(see Figure 4), which may indicate that the area is still looking for
maturity. This can also indicate that the area is growing, generating the need
to conduct new studies, especially experimental studies that can individually
assess the effects of different methods, tools, frameworks, and gamification
elements on people’s flow experience.

Still, on the growth of the area and the need for further studies, the results
of this systematic review identified that researchers from 19 countries (see
section 3) have been involved in research so far. However, continents like
Africa and Oceania (see Figure 3) have not yet started to produce research in
this regard. The result also indicates the importance of different countries
to carry out new studies or replicate studies already carried out in other
countries to analyze the data from different perspectives.

Not all studies have defined the methods and tools with the ultimate goal of
providing a flow experience to participants (i.e., in some studies, it is not
explicit what the purpose of the method or tools and whether it was designed
to lead participants to a flow experience). However, it is clear that
different methods and tools were used in the studies and that there is no
agreement regarding the use of any specific method or tool in the studies
about gamification and Flow Theory. Specifically about tools, because only two
studies make the tools used explicit, it is not possible to have insights
about the use of specific tools (e.g., intelligent tutors or virtual reality
glasses) in the studies on this domain.

A large variety of gamification methods have been used to provide users’ flow
experience in gamified settings. This indicates that, in general, research on
this subject is still in its initial phase and tend to grow in the coming
years, as there is still no consensus on which methods to use, and which are
the best gamification elements. Thus, we believe that new studies can be
conducted by evaluating and comparing different methods and elements of
gamification.

Although it is possible to identify which game elements were most used in the
selected studies, the studies do not describe whether each element was
specifically thought based on some dimension of the Flow Theory. The most used
game elements in the studies, generally correspond to the same game elements
used in the large gamification area (i.e., points, badges, and leaderboards).
However, some unusual game elements (e.g., narrative and curves) were also
used.

All studies that reported the method used to analyze the users’ flow
experience, used a questionnaire. This is explained by the fact that this
method is the simplest (e.g., methods for automatic identification based on
data analysis) [19], and other methods are more expensive or that cannot be
used massively (e.g., electroencephalograms or eye trackers) [46]. On the
other hand, it is surprising that there is no consensus regarding the
questionnaire used, and that many studies use questionnaires that have not
been empirically validated for the gamification domain (e.g., the
questionnaire validated by Hamari and Koivisto [35]).

Some studies identified that gamification positively affected the flow
experience of users, others demonstrated that the effects were negative, and
in other cases, no effects were perceived. This result is also similar to the
results found in other recent secondary studies on gamification [5, 47]. This
may have occurred due to two main facts: i) studies analyzing the effects of
gamification on the experience of people flow are still few, and ii) different
methods have been evaluated using different analysis techniques. In both
cases, the results highlight the importance of conducting further research,
assessing the effects of gamification on the users’ flow experience.

At the same time, it is difficult to analyze what affected the participants’
flow experience (e.g., method, tool, or element of gamification itself). This
difficulty is because, in general, the studies do not report, for example, how
each gamification element was used individually, nor was any moderating
intervention used to analyze these effects. Therefore, it is still difficult
to identify whether gamification alone affects people’s flow experience, and
what the exact effects of gamification are. Through our results, future
studies may conduct meta-analyzes to specifically analyze these effects. At
the same time, this result also draws attention to the importance that new
experimental studies, following strict criteria, should be conducted to
analyze, for example, the effects of specific gamification elements, on the
experience of the flow of participants in different types of scenarios.

The results of our secondary study allow us to begin to understand an overview
of the area, as well as to make room for new studies to start being produced
based on our results. Thus, based on our results, future studies may seek to
carry out quality analysis in the selected studies, that is, individually
analyze each of the 19 studies on different aspects and thereby have an
overview of the technical and scientific quality of the studies.

At the same time, to identify the individual effects of each method, tool, or
game element in the participants’ flow experience, a meta-analysis can be
conducted and consequently increase and deepen the results obtained in our
study. Through a meta-analysis, it will also be possible to reflect on future
actions that can be taken to deepen the area and even propose a research
agenda as well as in other recent studies in the area of gamification.

Finally, our results contribute directly to the area of gamification, and
indirectly to areas such as information systems, human-computer interaction,
and user-centered design, showing a state-of-the-art in studies that relate
gamification and Flow Theory, showing that the area has grown in recent years,
however, studies are still initial and deserve to be further developed, as
well as, that there is a great opening of space for the development of new
studies to propose and evaluate new gamified experiences to positively affect
people’s flow experience.

###  3.2 Limitations

Some inherent limitations to this type of study are present. In general, we
use mechanisms to mitigate all perceived limitations. Initially, the search
string for the study may not match all topics in the study area. To mitigate
this limitation, we use a specific method for defining search strings (i.e.,
PICOC method) and validate the string with domain experts. The process of
selecting studies and extracting data may have failed to include some studies.
To mitigate this limitation, in both phases, the process was conducted by two
experts. At the same time, in the case of disagreement, the experts debated
each specific case. The term gamification is often confused with games, which
may include studies that focus on games and not on gamified systems. To
mitigate possible limitations in this regard, we always follow what has been
defined by each study’s authors. In other words, if the authors said it was a
gamified application, we trust and consider it as gamification.

##  4 Concluding Remarks

Understanding how gamification affects people’ flow experience is a current
challenge in gamification studies. In this paper, we present the results of a
systematic literature review conducted to identify the state-of-the-art in the
use of gamification related to Flow Theory. Following a well-known process for
conducting secondary studies (e.g., systematic reviews), we found 19 studies
that were analyzed to answer our research questions.

The results of our study show that several different methods and gamification
elements have been used to get people into a flow experience. However, there
is no consensus as to which methods and gamification elements should be used.
There only is a consensus on using questionnaires to analyze the people’s flow
experience in gamified settings, to the detriment of other methods, such as
interviews and interaction data analysis. Still, there is no consensus on
which questionnaire to use. Finally, we identified that the results related to
the effects of gamification in the users’ flow experience are still
inconsistent.

The results directly contribute to understanding the effects of gamification
on the people’s flow experience and decision-making in conducting future
studies. As future studies, we suggest extending this systematic review by
answering other research questions (deeper analysis of the existing research,
quotation analysis, etc), analyzing the quality of the selected studies,
investigating potential research biases, and proposing a research agenda based
on the results. We also suggest that future studies invest in conducting meta-
analyses considering the studies included in this systematic review.

###### Acknowledgements.

The authors would like to thank the grant provided by São Paulo Research
Foundation (FAPESP), Projects: 2018/07688-1, 2020/02801-4, 2016/02765-2, and
2018/15917-0; CAPES and CNPq.

## References

  * Deterding et al. [2011] S. Deterding, D. Dixon, R. Khaled, L. Nacke,  From game design elements to gamefulness: defining gamification,  in: Proceedings of the 15th international academic MindTrek conference: Envisioning future media environments, ACM, 2011, pp. 9–15.
  * Dreimane [2019] S. Dreimane,  Gamification for education: Review of current publications,  in: Didactics of smart pedagogy, Springer, 2019, pp. 453–464.
  * Noorbehbahani et al. [2019] F. Noorbehbahani, F. Salehi, R. J. Zadeh,  A systematic mapping study on gamification applied to e-marketing,  Journal of Research in Interactive Marketing (2019).
  * Muangsrinoon and Boonbrahm [2019] S. Muangsrinoon, P. Boonbrahm,  Game elements from literature review of gamification in healthcare context.,  Journal of Technology and Science Education 9 (2019) 20–31.
  * Toda et al. [2017] A. M. Toda, P. H. Valle, S. Isotani,  The dark side of gamification: An overview of negative effects of gamification in education,  in: Researcher Links Workshop: Higher Education for All, Springer, 2017, pp. 143–156.
  * Koivisto and Hamari [2019] J. Koivisto, J. Hamari,  The rise of motivational information systems: A review of gamification research,  International Journal of Information Management 45 (2019) 191–210.
  * Oliveira et al. [2020] W. Oliveira, A. Toda, P. Toledo, L. Shi, J. Vassileva, I. I. Bittencourt, S. Isotani,  Does tailoring gamified educational systems matter? the impact on students’ flow experience,  Hawaii International Conference on System Sciences (2020).
  * Hamari et al. [2014] J. Hamari, J. Koivisto, H. Sarsa,  Does gamification work?–a literature review of empirical studies on gamification,  in: System Sciences (HICSS), 2014 47th Hawaii International Conference on System Sciences, IEEE, 2014, pp. 3025–3034.
  * Bai et al. [2020] S. Bai, K. F. Hew, B. Huang,  Is gamification “bullshit”? evidence from a meta-analysis and synthesis of qualitative data in educational contexts,  Educational Research Review (2020) 100322.
  * Csikszentmihalyi and Csikszentmihalyi [1975] M. Csikszentmihalyi, I. Csikszentmihalyi, Beyond boredom and anxiety, volume 721, Jossey-Bass San Francisco, 1975.
  * Csikszentmihalyi and Larson [2014] M. Csikszentmihalyi, R. Larson,  Validity and reliability of the experience-sampling method,  in: Flow and the foundations of positive psychology, Springer, 2014, pp. 35–54.
  * Hong et al. [2019] J.-C. Hong, C.-R. Tsai, H.-S. Hsiao, P.-H. Chen, K.-C. Chu, J. Gu, J. Sitthiworachart,  The effect of the “prediction-observation-quiz-explanation” inquiry-based e-learning model on flow experience in green energy learning,  Computers & Education 133 (2019) 127–138.
  * Huang et al. [2018] H.-C. Huang, T. T. L. Pham, M.-K. Wong, H.-Y. Chiu, Y.-H. Yang, C.-I. Teng,  How to create flow experience in exergames? perspective of flow theory,  Telematics and Informatics 35 (2018) 1288–1296.
  * Zhou [2019] T. Zhou,  The effect of flow experience on users’ social commerce intention,  Kybernetes (2019).
  * Kim and Ko [2019] D. Kim, Y. J. Ko,  The impact of virtual reality (vr) technology on sport spectators’ flow experience and satisfaction,  Computers in human behavior 93 (2019) 346–356.
  * Kitchenham [2004] B. Kitchenham,  Procedures for performing systematic reviews,  Keele, UK, Keele University 33 (2004) 1–26.
  * Klock et al. [2020] A. C. T. Klock, I. Gasparini, M. S. Pimenta, J. Hamari,  Tailored gamification: A review of literature,  International Journal of Human-Computer Studies 144 (2020) 102495. URL: http://www.sciencedirect.com/science/article/pii/S1071581920300975. doi:https://doi.org/10.1016/j.ijhcs.2020.102495.
  * Kitchenham and Charters [2007] B. Kitchenham, S. Charters,  Guidelines for performing systematic literature reviews in software engineering,  Citeseer (2007).
  * Oliveira et al. [2018] W. Oliveira, I. I. Bittencourt, S. Isotani, D. Dermeval, L. B. Marques, I. F. Silveira,  Flow theory to promote learning in educational systems: Is it really relevant?,  Brazilian Journal of Computers in Education 26 (2018) 29.
  * Pastushenko et al. [2020] O. Pastushenko, W. Oliveira, S. Isotani, T. Hruška,  A methodology for multimodal learning analytics and flow experience identification within gamified assignments,  in: Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems, 2020, pp. 1–9.
  * Sehoon [2020] K. Sehoon,  How a company’s gamification strategy influences corporate learning: A study based on gamified mslp (mobile social learning platform),  Telematics and Informatics (2020) 101505\.
  * Palmas et al. [2020] F. Palmas, J. Raith, et al.,  A novel approach to interactive dialogue generation based on natural language creation with context-free grammars and sentiment analysis,  in: 2020 IEEE 20th International Conference on Advanced Learning Technologies (ICALT), IEEE, 2020, pp. 79–83.
  * Quintas et al. [2020] A. Quintas, J.-C. Bustamante, F. Pradas, C. Castellar,  Psychological effects of gamified didactics with exergames in physical education at primary schools: Results from a natural experiment,  Computers & Education (2020) 103874\.
  * Shen et al. [2020] Y. S. Shen, H. C. Choi, M. Joppe, S. Yi,  What motivates visitors to participate in a gamified trip? a player typology using q methodology,  Tourism Management 78 (2020) 104074.
  * Chan et al. [2019] C.-K. Chan, H.-M. Leung, M.-W. Kung,  Understanding the effect of gamification of learning using flow theory,  in: Shaping the Future of Education, Communication and Technology, Springer, 2019, pp. 3–14.
  * Weng et al. [2019] Y. Weng, T. Shen, S. Chen, B. Xiao,  Gamification in local intangible cultural heritage museums for children: A case design,  in: International Conference on Human-Computer Interaction, Springer, 2019, pp. 233–245.
  * Högberg et al. [2019] J. Högberg, J. Hamari, E. Wästlund,  Gameful experience questionnaire (gamefulquest): an instrument for measuring the perceived gamefulness of system use,  User modeling and user-adapted interaction 29 (2019) 619–660.
  * Ilhan and Fietkiewicz [2019] A. Ilhan, K. J. Fietkiewicz,  Learning for a healthier lifestyle through gamification: a case study of fitness tracker applications,  in: Perspectives on Wearable Enhanced Learning (WELL), Springer, 2019, pp. 333–364.
  * Antonaci et al. [2017] A. Antonaci, D. Peter, R. Klemke, T. Bruysten, C. M. Stracke, M. Specht,  gmoocs–flow and persuasion to gamify moocs,  in: International Conference on Games and Learning Alliance, Springer, 2017, pp. 126–136.
  * Zhu et al. [2017] Y. Zhu, L. Pei, J. Shang,  Improving video engagement by gamification: A proposed design of mooc videos,  in: International Conference on Blended Learning, Springer, 2017, pp. 433–444.
  * Korn and Tietz [2017] O. Korn, S. Tietz,  Strategies for playful design when gamifying rehabilitation: a study on user experience,  in: Proceedings of the 10th International Conference on PErvasive Technologies Related to Assistive Environments, 2017, pp. 209–214.
  * Loos and Crosby [2017] L. A. Loos, M. E. Crosby,  Gamification methods in higher education,  in: International Conference on Learning and Collaboration Technologies, Springer, 2017, pp. 474–486.
  * Roh et al. [2016] S. Roh, K. Seo, J. Lee, J. Kim, H. B. Ryu, C. Jung, H. Lee, J. Shin,  Goal-based manufacturing gamification: bolt tightening work redesign in the automotive assembly line,  in: Advances in ergonomics of manufacturing: managing the enterprise of the future, Springer, 2016, pp. 293–304.
  * Shi et al. [2014] L. Shi, A. I. Cristea, S. Hadzidedic, N. Dervishalidovic,  Contextual gamification of social interaction–towards increasing motivation in social e-learning,  in: International Conference on Web-Based Learning, Springer, 2014, pp. 116–122.
  * Hamari and Koivisto [2014] J. Hamari, J. Koivisto,  Measuring flow in gamification: Dispositional flow scale-2,  Computers in Human Behavior 40 (2014) 133–143.
  * Sepehr and Head [2013] S. Sepehr, M. Head,  Competition as an element of gamification for learning: an exploratory longitudinal investigation,  in: Proceedings of the First International Conference on Gameful Design, Research, and Applications, 2013, pp. 2–9.
  * Ikehara et al. [2013] C. S. Ikehara, M. E. Crosby, P. A. Silva,  Combining augmented cognition and gamification,  in: International Conference on Augmented Cognition, Springer, 2013, pp. 676–684.
  * Korn et al. [2012] O. Korn, M. Brach, A. Schmidt, T. Hörz, R. Konrad,  Context-sensitive user-centered scalability: an introduction focusing on exergames and assistive systems in work contexts,  in: E-learning and games for training, education, health and sports, Springer, 2012, pp. 164–176.
  * Landers et al. [2019] R. N. Landers, G. F. Tondello, D. L. Kappen, A. B. Collmus, E. D. Mekler, L. E. Nacke,  Defining gameful experience as a psychological state caused by gameplay: Replacing the term ‘gamefulness’ with three distinct constructs,  International Journal of Human-Computer Studies 127 (2019) 81–94.
  * Jackson and Eklund [2002] S. A. Jackson, R. C. Eklund,  Assessing flow in physical activity: the flow state scale-2 and dispositional flow scale-2.,  Journal of sport & exercise psychology 24 (2002).
  * Agarwal and Prasad [1997] R. Agarwal, J. Prasad,  The role of innovation characteristics and perceived voluntariness in the acceptance of information technologies,  Decision sciences 28 (1997) 557–582.
  * IJsselsteijn et al. [2013] W. A. IJsselsteijn, Y. A. de Kort, K. Poels,  The game experience questionnaire,  Eindhoven: Technische Universiteit Eindhoven (2013) 3–9.
  * Van Exel and De Graaf [2005] J. Van Exel, G. De Graaf,  Q methodology: A sneak preview,  Rotterdam, Netherlands (2005).
  * Larson and Csikszentmihalyi [2014] R. Larson, M. Csikszentmihalyi,  The experience sampling method,  in: Flow and the foundations of positive psychology, Springer, 2014, pp. 21–34.
  * Mahfouz and Guo [2011] A. Y. Mahfouz, Y. M. Guo,  An overview of flow theory in ecommerce,  17th Americas Conference on Information Systems (2011).
  * Oliveira [2019] W. Oliveira,  Towards automatic flow experience identification in educational systems: A human-computer interaction approach,  in: Extended Abstracts of the Annual Symposium on Computer-Human Interaction in Play Companion Extended Abstracts, 2019, pp. 41–46.
  * Jarnac de Freitas and Mira da Silva [2020] M. Jarnac de Freitas, M. Mira da Silva,  Systematic literature review about gamification in moocs,  Open Learning: The Journal of Open, Distance and e-Learning (2020) 1–23.