
    \caption{Box plots of the six performance indicators achieved by the interaction evaluation for the ACO$_3$ algorithm in Setting 3.a, 3.b, and 3.c.}
    \label{fig:eval:wh:inter3_box}
\end{figure*}

\subsubsection{Comparison of NSGA-II and Random Planned Warehouses for $ACO_4$}
\label{sec:nsgaRandomAco4}
This setting repeats the Settings 3.a to 3.c for the $ACO_4$ algorithm.
We now discuss the results for the Settings 4.a to 4.c.
In the small warehouse, the Pareto front of the NSGA-II warehouse covers the entire reference Pareto front and the random warehouse fails to cover a single solution.
Thus, all solutions found in the random warehouse are dominated by the solutions of the NSGA-II warehouse.
The GD and ED values for the random warehouse are higher than the ones of the NSGA-II warehouse which shows that the Pareto front of the random warehouse is further away from the reference front. 
Hence, the solutions of the random warehouse are noticeably worse than the solutions found in the NSGA-II warehouse.
In the medium warehouse, the Pareto front of the NSGA-II warehouse does not always cover the entire reference front, while the random warehouse covers at least one solution in the reference front in 7 of 50 repetitions.
Thus, the $ACO_4$ variant occasionally finds a few solutions in the random warehouse that are comparable with the solutions found in the NSGA-II warehouse.
Despite these few outliers, the results show a similar behavior as in the previous setting.
Similar to the previous settings, the evaluation in the large warehouse show comparable results.
The NSGA-II warehouse covers all solutions in the reference front in 49 of 50 repetitions and the random warehouse manages to cover at least one solution in the reference front. 
In summary, the results show that the $ACO_4$ variant also finds better pick routes if the warehouse applies the NSGA-II storage strategy and both algorithms interact well with each other.

\begin{figure*}[htb!]
    \centering
    \includegraphics[width=0.8\textwidth]{Figures/pdf/wh_inter4_box.pdf}
    \caption{Box plots of the six performance indicators achieved by the interaction evaluation for the $ACO_4$ algorithm in Setting 4.a, 4.b, and 4.c.}
    \label{fig:eval:wh:inter4_box}
\end{figure*}


\subsubsection{Comparison of $ACO_3$ and $ACO_4$ on NSGA-II Planned Warehouses}
\label{sec:nsgaAco}
This section investigates which ACO variant performs better if the warehouse applies the NSGA-II storage strategy.
Both variants are applied on five randomly selected customer orders on all warehouse sizes~(Settings 5.a, 5.b, 5.c). 
In the small warehouse, the Pareto front of the $ACO_3$ algorithm covers approximately 80\% of the reference front, while $ACO_4$ covers only 66\%.
Both ACO variants converge well towards the reference front as indicated by the low GD and ED values and find approximately ten solutions per problem instance, and IGD values of both fronts are almost the same.
However, the GS values indicate, that the solutions of the $ACO_4$ variant have a better distribution than the ones of $ACO_3$.
In the medium warehouse, the Pareto front of $ACO_3$ covers approximately 69\% of the reference front, while the one from $ACO_4$ covers only~41\%, and thus, the solutions found by $ACO_4$ tend to be dominated by the ones from $ACO_3$.
The GD and ED metric indicate that the $ACO_3$ Pareto front converges better towards the reference front.
Similar to the small warehouse, the GS indicate, that solutions of the $ACO_4$ Pareto front are better distributed.
In the large warehouse, the $ACO_3$ dominates $ACO_4$ even more with regards to the Coverage metric.
Furthermore, the GD and ED values of $ACO_4$ increased, indicating that the distance between the solutions in $ACO_4$ Pareto front and the solutions in the reference front become larger.
Again, the solutions in the $ACO_4$ Pareto front have a slightly better distribution than the solutions in the $ACO_3$ Pareto front.
However, this time, $ACO_3$ achieves better IGD values, as $ACO_3$ covers large parts of the reference front.
In summary, we can state that with increasing warehouse size, the $ACO_3$ variant finds better pick routes than the $ACO_4$ variant while both variants find approximately the same number of solutions per problem instance.
However, the solutions found by the $ACO_4$ variant are slightly better distributed than the ones from the $ACO_3$ variant.


\begin{figure*}[htb!]
    \centering
    \includegraphics[width=0.8\textwidth]{Figures/pdf/wh_inter5_box.pdf}
    \caption{Box plots of the six performance indicators achieved by the interaction evaluation between $ACO_3$ and $ACO_4$ with the NSGA-II algorithm in Setting 5.a, 5.b, and 5.c.}
    \label{fig:eval:wh:inter5_box}
\end{figure*}



\subsection{Threats to Validity}
We identified the following threats to validity of our evaluation.
First, the NSGA-II and ACO algorithms are evaluated in three mezzanine warehouses of different sizes.
However, real-world mezzanine warehouses may consist of more floors, blocks, pick aisles, and racks than specified in the warehouses used for evaluation. 
Nevertheless, we are convinced that our defined warehouses form a representative set for mezzanine warehouses and can easily be extended for further evaluation runs.
Second, since the algorithms are evaluated in warehouses that apply either the random or the NSGA-II storage strategy, the evaluation results may not be transferable to warehouses that apply different storage strategies.
Even though the product assortment, the product correlations, the customer orders, and the storage allocations are randomly generated reflecting specific characteristics of real-world mezzanine warehouses, the proposed algorithms are easily transferable to real application data. 
Third, we decided to compare the ACO algorithm only to one order picking policy. 
This decision was made in awareness of the limited expressiveness of our results but was necessary as the majority of policies in the literature violate assumptions made in this work.
Finally, we only evaluate our NSGA-II and ACO algorithms against heuristic policies.
Hence, they also should be evaluated against other optimization methods like other evolutionary optimization algorithms or graph-based optimization techniques.
However, we decided to do this evaluations as future work.

\section{Conclusion}
\label{sec:conclusion}
Due to the complexity of the storage assignment and the order picking problem, efficient optimization algorithms are required to find satisfactory solutions within reasonable times.
This thesis proposes an NSGA-II algorithm for optimizing the storage assignment problem, and an ACO algorithm for optimizing the order picking problem in mezzanine warehouses.
The algorithms incorporate knowledge about the interdependency between both problems to improve the overall warehouse performance.
Besides optimizing economic constraints, the algorithms also optimize ergonomic criteria, as mezzanine warehouses represent labor-intensive working environments in which the employees account for a large part of the warehouse performance. 
We evaluate the NSGA-II algorithm against three storage assignment policies frequently applied in practice: the random, the closest open location, and the rank-based policy.
The evaluation results show that the NSGA-II algorithm outperforms the alternatives already in smaller warehouses and the larger the warehouse, the better the NSGA-II algorithm prevails against the alternative policies.
We evaluate the ACO algorithm against the S-Shape heuristic that is frequently applied in practice.
Our evaluation results show that the ACO outperforms the S-Shape heuristic in all tested warehouse sizes.
Finally, we evaluate the interaction between the NSGA-II and the ACO algorithm.
The evaluation results show that both ACO variants find better pick routes if the warehouse assigns its products by applying the NSGA-II algorithm instead of the random storage strategy, thus, the NSGA-II and the ACO algorithm interact well with each other.

In the future, we plan to integrate additional features to further increase the applicability of the storage assignment.
First, we want to allow state changes of the mezzanine warehouses while the storage assignment is running which would make some of the solutions in the Pareto front infeasible.
Further, we plan to parallelize the storage assignment algorithm so that the sequential assignment of products is replaced and the execution times will decrease.
Regarding the order picking algorithm, we also aim at parallelizing the execution to reduce the required calculation times. 
Finally, we want to research on integrating forecasts of future assignment and order picking tasks to proactively replace goods within the storage that will be ordered in the near future. 

