faults.

The rest of the paper is organized as follows: In section \[sec:background\], we present the background of the proposed research. We present elements of scalable DL algorithms in section \[sec:motivation\] and a solution space for fault tolerant MaTEx-Caffe in section \[sec:design\]. In section \[sec:existing\_proposals\] we present existing features and proposals for fault tolerance in MPI and provide implementation details in section \[sec:implDetail\]. We discuss experimental results in section \[sec:exp\], related work in section \[sec:related\] followed by conclusions in section \[sec:conclusions\].

Background {#sec:background}
==========

In this section, we provide a brief overview of the proposed research. Specifically, we focus on deep neural networks (DNNs).

Deep Neural Networks
--------------------

       Symbol                    Meaning                                    
  ---------------- ----------------------------------- -- -- -- -- -- -- -- --
    $W^{(\ell)}$         weights of layer $\ell$                            
    $b^{(\ell)}$         biases of layer $\ell$                             
   $z^{(\ell+1)}$   $W^{(\ell)}a^{(\ell)}+b^{(\ell)}$                       
    $a^{(\ell)}$          $\sigma(z^{(\ell)})$                              
      $\sigma$        $\mathrm{ReLU}(x)=\max(0, x)$                         
      $n_\ell$              number of layers                                

  : Symbols for Backpropagation.[]{data-label="table:notation"}

\

A dataset is a collection of samples. Samples are often images, speech, text or raw vectors of numbers. ML algorithms typically split a dataset into a [*training set*]{}, used for learning the details of a model; a [*validation set*]{}, used to prevent overfitting and to tune hyper-parameters; and a [*testing set*]{}, used for the 