<p>you can add l1 and l2 Regularization :</p>
<pre><code>kernel_regularizer=keras.regularizers.l2(0.01))
</code></pre>
<p>you can add dropout :</p>
<pre><code>dropout = tf.keras.layers.Dropout(0.2)
</code></pre>
<p>Here, you can also try out different variants of relu like, ELU, LeakyReLU</p>
<p>Monte Carlo (MC) Dropout also you can try.
visit <a href="https://arxiv.org/abs/1506.02142" rel="nofollow noreferrer">https://arxiv.org/abs/1506.02142</a></p>
<p>try</p>
<pre><code>kernel_initializer=&quot;he_uniform&quot; 
# or
kernel_initializer=&quot;he_normal&quot;
</code></pre>
Issue with overfitting an ANN -- perhaps due to OneHotEncoder problem
<p>I've been working on building a simple ANN, as I'm a novice, and while I no longer receive any errors, after the first epoch, the accuracy jumps to 1.0, which suggests over fitting. I made sure that the dependent variable (y) column wasn't in both the X and y arrays.</p>
<p>The one hot encoder class I got from another post on stackoverflow, and feel that perhaps I haven't implemented it correctly, and that's causing issue.</p>
<p>I was thinking there may be an issue with this line calling the class, but am not 100% sure.</p>
<p>Any clarification on how to solve the over fitting, or to improve the model would be appreciated.</p>
<p>And lastly, for context: the df (found <a href="https://www.kaggle.com/psvishnu/bank-direct-marketing" rel="nofollow noreferrer">https://www.kaggle.com/psvishnu/bank-direct-marketing</a>)  contains data on 7 independent variables that describe a banks customer profile - ei, salary, number of credit cards, debt, etc</p>
<p>And the y = if the client subscribed a term deposit? (binary: &quot;yes&quot;,&quot;no&quot;)</p>
<pre><code>X = MultiColumnLabelEncoder(columns = ['housing', 'loan', 'default']).fit_transform(df_ann)
</code></pre>
<p>Full code snippet</p>
<pre><code>import numpy as np
import pandas as pd
import tensorflow as tf

#Importing Dataset
df = pd.read_csv('bank-full.csv', sep=';')
df_ann = df[['age', 'job', 'marital', 'education', 'default', 'balance', 'housing', 'loan', 'y']]

X = df_ann.iloc[:, :7].values
y = df_ann.iloc[:, -1].values


#Encoding Categorical Data
from sklearn.preprocessing import LabelEncoder
from sklearn.pipeline import Pipeline
le = LabelEncoder()

y[:] = le.fit_transform(y[:])


class MultiColumnLabelEncoder:
    def __init__(self,columns = None):
        self.columns = columns # array of column names to encode

    def fit(self,X,y=None):
        return self # not relevant here
   
        '''
        Transforms columns of X specified in self.columns using
        LabelEncoder(). If no columns specified, transforms all
        columns in X.
       '''

    def transform(self,X):
      
        output = X.copy()
        if self.columns is not None:
            for col in self.columns:
                output[col] = LabelEncoder().fit_transform(output[col])
        else:
            for colname,col in output.iteritems():
                output[colname] = LabelEncoder().fit_transform(col)
        return output

    def fit_transform(self,X,y=None):
        return self.fit(X,y).transform(X)

X = MultiColumnLabelEncoder(columns = ['housing','loan', 'default']).fit_transform(df_ann)


#OneHotEncoder (3d+)

from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import OneHotEncoder

ct = ColumnTransformer(transformers=[('encoder', OneHotEncoder(handle_unknown='ignore'), ['job', 'marital', 'education'])], remainder='passthrough')
X = np.array(ct.fit_transform(X))

X = np.asarray(X).astype('float32')
y= np.asarray(y).astype('float32')

#Split to training/test
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 0)

#Feature Scaling
from sklearn.preprocessing import StandardScaler
sc = StandardScaler()
X_train = sc.fit_transform(X_train)
X_test = sc.transform(X_test)



# Building the ANN

#Initializing
ann = tf.keras.models.Sequential()

#Adding Input Layer 
ann.add(tf.keras.layers.Dense(units=6, activation='relu'))

#Second Hidden Layer
ann.add(tf.keras.layers.Dense(units=6, activation='relu'))

#Output Layer
ann.add(tf.keras.layers.Dense(units=1, activation='sigmoid'))

#Training ANN
ann.compile(optimizer = 'adam', loss = 'binary_crossentropy', metrics = ['accuracy'])

ann.fit(X_train, y_train, batch_size = 32, epochs = 200)


#Predicting Test Results
y_pred = ann.predict(X_test)
y_pred = (y_pred &gt; 0.5)
print(np.concatenate((y_pred.reshape(len(y_pred),1), y_test.reshape(len(y_test),1)),1))


#Confusion Matrix
from sklearn.metrics import confusion_matrix, accuracy_score
cm = confusion_matrix(y_test, y_pred)
print(cm)
accuracy_score(y_test, y_pred)


</code></pre>