<p>@Josh Payne's answer is correct, but I'll expand on it for those who want to use the .mat file with an emphasis on typical data splits.</p>

<p>The data itself has already been split up in to a training and test set. Here's how I accessed the data:</p>

<pre><code>    from scipy import io as sio
    mat = sio.loadmat('emnist-letters.mat')
    data = mat['dataset']

    X_train = data['train'][0,0]['images'][0,0]
    y_train = data['train'][0,0]['labels'][0,0]
    X_test = data['test'][0,0]['images'][0,0]
    y_test = data['test'][0,0]['labels'][0,0]
</code></pre>

<p>There is an additional field 'writers' (e.g. <code>data['train'][0,0]['writers'][0,0]</code>) that distinguishes the original sample writer. Finally, there is another field <code>data['mapping']</code>, but I'm not sure what it is mapping the digits to.</p>

<p>In addition, in Secion II D, the <a href="https://arxiv.org/abs/1702.05373v1" rel="nofollow noreferrer">EMNIST paper</a> states that "the last portion of the training set, equal in size to the testing set, is set aside as a validation set". Strangely, the .mat file  training/testing size does not match the number listed in Table II, but it does match the size in Fig. 2.</p>

<pre><code>    val_start = X_train.shape[0] - X_test.shape[0]
    X_val = X_train[val_start:X_train.shape[0],:]
    y_val = y_train[val_start:X_train.shape[0]]
    X_train = X_train[0:val_start,:]
    y_train = y_train[0:val_start]
</code></pre>

<p>If you don't want a validation set it is fine to leave these samples in the training set.</p>

<p>Also, if you would like to reshape the data into 2D, 28x28 sized images instead of a 1D 784 array, to get the correct image orientation you'll need to do a numpy reshape using Fortran ordering (Matlab uses column-major ordering, just like Fortran. <a href="https://en.wikipedia.org/wiki/Row-_and_column-major_order" rel="nofollow noreferrer">reference</a>). e.g. -</p>

<pre><code>    X_train = X_train.reshape( (X_train.shape[0], 28, 28), order='F')
</code></pre>
loading EMNIST-letters dataset
<p>I have been trying to find a way to load the EMNIST-letters dataset but without much success. I have found interesting stuff in the structure and can't wrap my head around what is happening. Here is what I mean:</p>

<p>I downloaded the .mat format <a href="https://www.nist.gov/itl/iad/image-group/emnist-dataset" rel="noreferrer">in here</a></p>

<p>I can load the data using </p>

<pre><code>import scipy.io
mat = scipy.io.loadmat('letter_data.mat') # renamed for conveniance
</code></pre>

<p>it is a dictionnary with the keys as follow:</p>

<pre><code>dict_keys(['__header__', '__version__', '__globals__', 'dataset'])
</code></pre>

<p>the only key with interest is dataset, which I havent been able to gather data from. printing the shape of it give this:</p>

<pre><code>&gt;&gt;&gt;print(mat['dataset'].shape)
(1, 1)
</code></pre>

<p>I dug deeper and deeper to find a shape that looks somewhat like a real dataset and came across this:</p>

<pre><code>&gt;&gt;&gt;print(mat['dataset'][0][0][0][0][0][0].shape)
(124800, 784)
</code></pre>

<p>which is exactly what I wanted but I cant find the labels nor the test data, I tried many things but cant seem to understand the structure of this dataset.</p>

<p>If someone could tell me what is going on with this I would appreciate it</p>