#How to prevent Gradient Vanishing <-> Exploding?

#1. ReLU and Transform of ReLU

#2. Weight initialization
#1) Xavier Initialization(Glorot Initialization)
#2) He initialization

#3. Batch Normalization
#1) Internal Covariate Shift
#2) Batch Normalization(- too depending on size of mini batch & too dificult to apply on RNN)

#4. Layer Normalization

#How to prevent Gradient Vanishing <-> Exploding?

#1. ReLU and Transform of ReLU

#2. Weight initialization
#1) Xavier Initialization(Glorot Initialization)
#2) He initialization

#3. Batch Normalization
#1) Internal Covariate Shift
#2) Batch Normalization(- too depending on size of mini batch & too dificult to apply on RNN)

#4. Layer Normalization

#How to prevent Gradient Vanishing <-> Exploding? #1. ReLU and Transform of ReLU #2. Weight initialization #1) Xavier Initialization(Glorot Initialization) #2) He initialization #3. Batch Normalization #1) Internal Covariate Shift #2) Batch Normalization(- too depending on size of mini batch & too dificult to apply on RNN) #4. Layer Normalization