learn complex patterns]{}            [Able to learn complex data patterns]{}

  : Comparison between L1 and L2[]{data-label="Table 1"}

In addition to the abovementioned methods, data augmentation [@wang17] and dropout [@srivastava14] are commonly used regularization techniques. Data augmentation is the most straightforward, especially for image classification and object detection. The data augmentation method creates and adds fake data to the training dataset to avoid overfitting. Operations including image transformation, cropping, flipping, color adjustment, and image rotation can improve the generalization in different cases [@devries17]. For documentation recognition [@lecun98], several transformations for enhanced model accuracy and robustness are mainly applied. For character recoginition task [@bengio11] image transformations and occlusions for their handwritten characters were combined.\

![Comparison of a neural network before and after dropout is applied [@srivastava14] []{data-label="Figure 7"}](7.jpg)

Dropout is a technique used to select neurons randomly with a given probability that are not used during training, which makes the network less sensitive to the specific weights of neurons [@hinton12a]. As a result, the simplified version of a neuron network can reduce overfitting (Figure \[Figure 7\]). The probability of neuron deactivation applied for each weight-updated step is suggested to be around 20%-50%. An overlarge dropout rate will over-simplify the model, whereas a small 