use syntactic information when decoding language, which the model discards. Yet, for many tasks they achieve state of the art performance. We compare our BoW to a character-level CNN on gender prediction, our most data rich problem. A character-level CNN is well suited to large amounts of messy, user generated data. Pooling layers in a CNN allow generalization of words like “gooooooooo” and “gooooooo”, while BoW must learn distinct weights. Surprisingly, the CNN does not outperform the simple BoW as shown in Table \[acc\_gender\].

We found the choice of prediction model is not as important as preprocessing. In initial experiments, Support Vector Machines [@suykens1999least] and logistic regression, and $\ell_2$ regularized regression yielded similar performance, depending on choice of $n$-grams and whether Singular Value Decomposition was used [@golub1970singular]. We implement ridge regression and classification for simplicity.

Inferring human traits from social media is now being done using deep models [@iyyer2014political; @preoctiuc2017beyond]. That may be useful in some cases, but for this project the deep model offered no performance boost or intuition to underlying human behavior. Perhaps a continuous bag of words [@mikolov2013distributed] and recurrent neural network [@felbo2017using] would have done better, but researchers should not consider deep learning essential for this 