title: What activation function on the last layer and loss function should I use in an auto encoder for reconstructing a sequence of events? [Keras]
body: <p>My data set is a 3D array of the size (M,t,N) where M is the number of samples, t is the number of timesteps in a sequence and N is the number of possible events that can happen at time t. By selecting a specific M we have a 2D array of size (t,N) where each row is a timestep and each column is an event. Each column is set to 1 if that event happened at time t, otherwise it's set to 0. Only 1 event can happen at any given timestep.</p>
  <p>I want to try and build an auto-encoder for anomaly detection, and in the tutorials and blogs I have read, the last activation layer is 'relu' and the loss function is 'mse'. But since I am trying to basically reconstruct a classification with N classes, would 'softmax' as the last layer and 'categorical_crossentropy' be better?</p>
  <pre><code>inputs = Input(shape = (timesteps,n_features))
  # Encoder
  lstm_enc_1 = LSTM(32, activation='relu', input_shape=(timesteps, n_features), return_sequences=True)(inputs)
  lstm_enc_2 = LSTM(latent_dim, activation='relu', return_sequences=False)(lstm_enc_1)
  repeater = RepeatVector(timesteps)
  # Decoder
  lstm_dec_1 = LSTM(latent_dim, activation='relu', return_sequences=True)
  lstm_dec_2 = LSTM(32, activation='relu', return_sequences=True)
  time_dis = TimeDistributed(Dense(n_features,activation='softmax')) #&lt;-- Does this make sense here?
  
  z = repeater(lstm_enc_2)
  h = lstm_dec_1(z)
  decoded_h = lstm_dec_2(h)
  decoded = time_dis(decoded_h)
  
  ae = Model(inputs,decoded)
  ae.compile(loss='categorical_crossentropy', optimizer='adam') #&lt;-- Does this make sense here?
  </code></pre>
  <p>Or should I, for some reason, still use 'relu' and 'mse' as the last activation function and loss function?</p>
  <p>Any input is appreciated.</p>