title: How do I load in the MNIST digits and label data in MATLAB?
body: <p>I am trying to run the code given in the link</p>
  
  <p><a href="https://github.com/bd622/DiscretHashing" rel="nofollow noreferrer">https://github.com/bd622/DiscretHashing</a></p>
  
  <p>Discrete Hashing is a method for dimensionality reduction that is used on approximate nearest neighbor search. I want to load in the implementation on the MNIST database that is available in <a href="http://yann.lecun.com/exdb/mnist/" rel="nofollow noreferrer">http://yann.lecun.com/exdb/mnist/</a>.  I have extracted the files from their compressed gz format.</p>
  
  <p><strong>PROBLEM 1 :</strong></p>
  
  <p>Using the solution to read MNIST database provided in <a href="https://stackoverflow.com/questions/24127896/reading-mnist-image-database-binary-file-in-matlab">Reading MNIST Image Database binary file in MATLAB</a></p>
  
  <p>I am getting the following error:</p>
  
  <pre><code>Error using fread
  Invalid file identifier.  Use fopen to generate a valid file identifier.
  
  Error in Reading (line 7)
  A = fread(fid, 1, 'uint32');
  </code></pre>
  
  <p>Here is the code:</p>
  
  <pre><code>clear all;
  close all;
  
  %//Open file
  fid = fopen('t10k-images-idx3-ubyte', 'r');
  
  A = fread(fid, 1, 'uint32');
  magicNumber = swapbytes(uint32(A));
  
  %//For each image, store into an individual cell
  imageCellArray = cell(1, totalImages);
  for k = 1 : totalImages
      %//Read in numRows*numCols pixels at a time
      A = fread(fid, numRows*numCols, 'uint8');
      %//Reshape so that it becomes a matrix
      %//We are actually reading this in column major format
      %//so we need to transpose this at the end
      imageCellArray{k} = reshape(uint8(A), numCols, numRows)';
  end
  
  %//Close the file
  fclose(fid);
  </code></pre>
  
  <p><strong>UPDATE : Problem 1 solved and the revised code is</strong></p>
  
  <pre><code>clear all;
  close all;
  
  %//Open file
  fid = fopen('t10k-images.idx3-ubyte', 'r');
  
  A = fread(fid, 1, 'uint32');
  magicNumber = swapbytes(uint32(A));
  
  %//Read in total number of images
  %//A = fread(fid, 4, 'uint8');
  %//totalImages = sum(bitshift(A', [24 16 8 0]));
  
  %//OR
  A = fread(fid, 1, 'uint32');
  totalImages = swapbytes(uint32(A));
  
  %//Read in number of rows
  %//A = fread(fid, 4, 'uint8');
  %//numRows = sum(bitshift(A', [24 16 8 0]));
  
  %//OR
  A = fread(fid, 1, 'uint32');
  numRows = swapbytes(uint32(A));
  
  %//Read in number of columns
  %//A = fread(fid, 4, 'uint8');
  %//numCols = sum(bitshift(A', [24 16 8 0]));
  
  %// OR
  A = fread(fid, 1, 'uint32');
  numCols = swapbytes(uint32(A));
  
  for k = 1 : totalImages
      %//Read in numRows*numCols pixels at a time
      A = fread(fid, numRows*numCols, 'uint8');
      %//Reshape so that it becomes a matrix
      %//We are actually reading this in column major format
      %//so we need to transpose this at the end
      imageCellArray{k} = reshape(uint8(A), numCols, numRows)';
  end
  
  %//Close the file
  fclose(fid);
  </code></pre>
  
  <p><strong>PROBLEM 2:</strong></p>
  
  <p>I cannot understand how to apply the 4 files of MNIST in the code. The code contains variables</p>
  
  <pre><code>traindata = double(traindata);
  testdata = double(testdata);
  </code></pre>
  
  <p>How do I prepare the MNIST database so that I can apply to the implementation?</p>
  
  <p>UPDATE : I implemented the solution but I keep getting this error</p>
  
  <pre><code>Error using fread
  Invalid file identifier.  Use fopen to generate a valid file identifier.
  
  Error in mnist_parse (line 11)
  A = fread(fid1, 1, 'uint32');
  </code></pre>
  
  <p>These are the files</p>
  
  <p><code>demo.m</code> % this is the main file that calls the function to read in the MNIST data</p>
  
  <pre><code>clear all
  clc
  [Trainimages, Trainlabels] = mnist_parse('C:\Users\Desktop\MNIST\train-images-idx3-ubyte', 'C:\Users\Desktop\MNIST\train-labels-idx1-ubyte');
  
  [Testimages, Testlabels] = mnist_parse('t10k-images-idx3-ubyte', 't10k-labels-idx1-ubyte');
  
  k=5;
  digit = images(:,:,k);
  lbl = label(k);
  </code></pre>
  
  <hr>
  
  <pre><code> function [images, labels] = mnist_parse(path_to_digits, path_to_labels)
  
  % Open files
  fid1 = fopen(path_to_digits, 'r');
  
  % The labels file
  fid2 = fopen(path_to_labels, 'r');
  
  % Read in magic numbers for both files
  A = fread(fid1, 1, 'uint32');
  magicNumber1 = swapbytes(uint32(A)); % Should be 2051
  fprintf('Magic Number - Images: %d\n', magicNumber1);
  
  A = fread(fid2, 1, 'uint32');
  magicNumber2 = swapbytes(uint32(A)); % Should be 2049
  fprintf('Magic Number - Labels: %d\n', magicNumber2);
  
  % Read in total number of images
  % Ensure that this number matches with the labels file
  A = fread(fid1, 1, 'uint32');
  totalImages = swapbytes(uint32(A));
  A = fread(fid2, 1, 'uint32');
  if totalImages ~= swapbytes(uint32(A))
      error('Total number of images read from images and labels files are not the same');
  end
  fprintf('Total number of images: %d\n', totalImages);
  
  % Read in number of rows
  A = fread(fid1, 1, 'uint32');
  numRows = swapbytes(uint32(A));
  
  % Read in number of columns
  A = fread(fid1, 1, 'uint32');
  numCols = swapbytes(uint32(A));
  
  fprintf('Dimensions of each digit: %d x %d\n', numRows, numCols);
  
  % For each image, store into an individual slice
  images = zeros(numRows, numCols, totalImages, 'uint8');
  for k = 1 : totalImages
      % Read in numRows*numCols pixels at a time
      A = fread(fid1, numRows*numCols, 'uint8');
  
      % Reshape so that it becomes a matrix
      % We are actually reading this in column major format
      % so we need to transpose this at the end
      images(:,:,k) = reshape(uint8(A), numCols, numRows).';
  end
  
  % Read in the labels
  labels = fread(fid2, totalImages, 'uint8');
  
  % Close the files
  fclose(fid1);
  fclose(fid2);
  
  end
  </code></pre>