We apply multi-task learning to image classification tasks on MNIST-like datasets. MNIST dataset has been referred to as the {\em drosophila} of machine learning and has been the testbed of many learning theories. The NotMNIST dataset and the FashionMNIST dataset have been created with the MNIST dataset as reference. In this work, we exploit these MNIST-like datasets for multi-task learning. The datasets are pooled together for learning the parameters of joint classification networks. Then the learned parameters are used as the initial parameters to retrain disjoint classification networks. The baseline recognition model are all-convolution neural networks. Without multi-task learning, the recognition accuracies for MNIST, NotMNIST and FashionMNIST are 99.56\%, 97.22\% and 94.32\% respectively. With multi-task learning to pre-train the networks, the recognition accuracies are respectively 99.70\%, 97.46\% and 95.25\%. The results re-affirm that multi-task learning framework, even with data with different genres, does lead to significant improvement.
 Multi-task learning BID1 ) enjoys the idea of pooling information that can be learned from data collected for multiple related tasks. Multiple sources of information can stem from multiple datasets, or even a single dataset, for multiple tasks. In this work, we focus on the case of using multiple datasets for multiple tasks. Namely, we use MNIST, FashionMNIST, and NotMNIST image datasets collected for digit recognition, fashion item recognition, and letter recognition, respectively.Information sharing in multi-task training can be achieved in various formality. For neural-network based deep learning, the sharing can happen at the input layer, the hidden layers, or the output layer. Input-layer multi-tasking combines heterogeneous input data, hidden-layer multi-tasking shares multiple groups of hidden layer units, and output-layer multi-tasking pools multiple output groups of categories. The implementation of a multi-task learning system depends on the data and the tasks at hand.Multi-task learning has been successfully applied to many applications of machine learning, from natural language processing BID4 ) and speech recognition BID5 ) to computer vision ) and drug discovery BID7 ). A recent review of multi-task learning in deep learning can be found in BID9 ). In this paper, we use multi-task learning in pre-training an all-convolution neural network model. We pass the parameters of trained multi-task models to single-task models. Evaluation on MNISTlike datasets show that using multi-task learning can improve image recognition accuracy. The more data we use, the better results we get. This agrees with statistical learning theory that using more data reduces the generalization gap, thus improving test set performance, even if the data comes from a different domain. The classification tasks of the images of digits, letters, and fashion items share parts of their hierarchical representations. By multi-task learning, it is possible to make such common representation robust to help individual classification tasks.Figure 6: Visualization of data manifolds with t-SNE. The left column is the case with multi-task learning, and the right column is the case without multi-task learning.
multi-task learning works  This paper presents a multi-task neural network for classification on MNIST-like datasets