<p><strong>In short:</strong> NNs are rarely the best models for classifying either small amounts data or the data that is already compactly represented by a few non-heterogeneous columns. Often enough, boosted methods or GLM would produce better results from a similar amount of effort.</p>
<p><strong>What can you do with your model?</strong> Counterintuitively, sometimes hindering the network capacity can be beneficial, especially when the number of network parameters exceeds number of training points. One can reduce the number of neurons, like in your case setting layer sizes to 16 or so and simultaneously removing layers; introduce regularizations (label smoothing, weight decay, etc); or generate more data by adding more derived columns in different (log, binary) scales.</p>
<p>Another approach would be to search for NNs models designed for your type of data. As, for example, <a href="https://arxiv.org/abs/1706.02515" rel="nofollow noreferrer">Self-Normalizing Neural Networks</a> or <a href="https://arxiv.org/abs/1606.07792" rel="nofollow noreferrer">Wide &amp; Deep Learning for Recommender Systems</a>.</p>
<p>If you get to try only 1 thing, I would recommend doing a grid search of the learning rate or trying a few different optimizers.</p>
<p><strong>How to make a better decision about which model to use?</strong> Look through finished kaggle.com competitions and find datasets similar to the one at hand, then check out the techniques used by the top places.</p>
keras accuracy doesn't improve more than 59 percent
<p>Here is the code I tried:</p>
<pre><code># normalizing the train data
cols_to_norm = [&quot;WORK_EDUCATION&quot;, &quot;SHOP&quot;, &quot;OTHER&quot;,'AM','PM','MIDDAY','NIGHT', 'AVG_VEH_CNT', 'work_traveltime', 'shop_traveltime','work_tripmile','shop_tripmile', 'TRPMILES_sum',
                'TRVL_MIN_sum', 'TRPMILES_mean', 'HBO', 'HBSHOP', 'HBW', 'NHB', 'DWELTIME_mean','TRVL_MIN_mean', 'work_dweltime', 'shop_dweltime', 'firsttrip_time', 'lasttrip_time']
dataframe[cols_to_norm] = dataframe[cols_to_norm].apply(lambda x: (x - x.min()) / (x.max()-x.min()))
# labels    
y = dataframe.R_SEX.values
</code></pre>
<hr />
<pre><code># splitting train and test set
X_train, X_test, y_train, y_test =train_test_split(X, y, test_size=0.33, random_state=42)

model = Sequential()
model.add(Dense(256, input_shape=(X_train.shape[1],), activation='relu'))
model.add(Dense(256, activation='relu'))
model.add(layers.Dropout(0.3))
model.add(Dense(256, activation='relu'))
model.add(layers.Dropout(0.3))
model.add(Dense(1, activation='sigmoid'))

model.compile(loss='binary_crossentropy', optimizer='adam' , metrics=['acc'])
print(model.summary())

model.fit(X_train, y_train , batch_size=128, epochs=30, validation_split=0.2)
</code></pre>
<hr />
<pre><code>Epoch 23/30
1014/1014 [==============================] - 4s 4ms/step - loss: 0.6623 - acc: 0.5985 - val_loss: 0.6677 - val_acc: 0.5918
Epoch 24/30
1014/1014 [==============================] - 4s 4ms/step - loss: 0.6618 - acc: 0.5993 - val_loss: 0.6671 - val_acc: 0.5925
Epoch 25/30
1014/1014 [==============================] - 4s 4ms/step - loss: 0.6618 - acc: 0.5997 - val_loss: 0.6674 - val_acc: 0.5904
Epoch 26/30
1014/1014 [==============================] - 4s 4ms/step - loss: 0.6614 - acc: 0.6001 - val_loss: 0.6669 - val_acc: 0.5911
Epoch 27/30
1014/1014 [==============================] - 4s 4ms/step - loss: 0.6608 - acc: 0.6004 - val_loss: 0.6668 - val_acc: 0.5920
Epoch 28/30
1014/1014 [==============================] - 4s 4ms/step - loss: 0.6605 - acc: 0.6002 - val_loss: 0.6679 - val_acc: 0.5895
Epoch 29/30
1014/1014 [==============================] - 4s 4ms/step - loss: 0.6602 - acc: 0.6009 - val_loss: 0.6663 - val_acc: 0.5932
Epoch 30/30
1014/1014 [==============================] - 4s 4ms/step - loss: 0.6597 - acc: 0.6027 - val_loss: 0.6674 - val_acc: 0.5910
&lt;tensorflow.python.keras.callbacks.History at 0x7fdd8143a278&gt;
</code></pre>
<p>I have tried modifying the neural network and double-cheking the data.</p>
<p>Is there anything I can do to improve the outcome? Is the model not deep enough? Is there any alternative models suited for my data? Does this mean these features have no predictive value? I'm kind of confused what to do next.</p>
<p>thank you</p>
<p><strong>Update:</strong></p>
<p>I tried adding new column do my dataframe which is the outcome of a KNN model for sex classification. Here is what I did:</p>
<pre><code>#Import knearest neighbors Classifier model
from sklearn.neighbors import KNeighborsClassifier

#Create KNN Classifier
knn = KNeighborsClassifier(n_neighbors=41)

#Train the model using the training sets
knn.fit(X, y)

#predict sex for the train set so that it can be fed to the nueral net
y_pred = knn.predict(X)

#add the outcome of knn to the train set
X = X.assign(KNN_result=y_pred)
</code></pre>
<p>It improved the training and validation accuracy up to 61 percent.</p>
<pre><code>Epoch 26/30
1294/1294 [==============================] - 8s 6ms/step - loss: 0.6525 - acc: 0.6166 - val_loss: 0.6604 - val_acc: 0.6095
Epoch 27/30
1294/1294 [==============================] - 8s 6ms/step - loss: 0.6523 - acc: 0.6173 - val_loss: 0.6596 - val_acc: 0.6111
Epoch 28/30
1294/1294 [==============================] - 8s 6ms/step - loss: 0.6519 - acc: 0.6177 - val_loss: 0.6614 - val_acc: 0.6101
Epoch 29/30
1294/1294 [==============================] - 8s 6ms/step - loss: 0.6512 - acc: 0.6178 - val_loss: 0.6594 - val_acc: 0.6131
Epoch 30/30
1294/1294 [==============================] - 8s 6ms/step - loss: 0.6510 - acc: 0.6183 - val_loss: 0.6603 - val_acc: 0.6103
&lt;tensorflow.python.keras.callbacks.History at 0x7fe981bbe438&gt;
</code></pre>
<p>Thank you</p>