the ratio between an estimated and the actual contaminate rate (or vice versa). Let $y$ be the true containment rate, and $\hat{y}$ the estimated rate, then the q-error is defined as follows. $$q-error(y,\hat{y})\ =\ \hat{y} > y\ ?\ \frac{\hat{y}}{y}\ :\ \frac{y}{\hat{y}}$$

In addition to optimizing the mean q-error, we also examined the mean squared error (MSE) and the mean absolute error (MAE) as optimization goals. MSE and MAE would optimize the squared/absolute differences between the predicted and the actual containment rates. Optimizing with theses metrics makes the model put less emphasis on heavy outliers (that lead to large errors). Therefore, we decided to optimize our model using the q-error metric which yielded better results.

Training and Testing Interface
------------------------------

Building CRN involves two main steps. (1) Generating a random training set using the schema and data information as described in Section \[Defining the database and the development set\]. (2) Repeatedly using this training data, we train the CRN model as described in Section \[Model\] until the mean q-error of the validation test starts to converges to its best absolute value. That is, we use the early stopping technique [@earlyStopping] and stop the training before convergence to avoid over-fitting. Both steps 