title: Check for differences between two (large) files
body: <p>I want to write a relatively simple program, that can backup files from my computer to a remote location and encrypt them in the process, while also computing a diff (well not really...I'm content with seeing if anything changed at all, not so much what has changed) between the local and the remote files to see which ones have changed and are necessary to update.</p>
  
  <p><em>I am aware that there are perfectly good programs out there to do this (rsync, or others based on duplicity). I'm not trying to reinvent the wheel, it's just supposed to be a learning experience for myself</em></p>
  
  <p>My question is regarding to the <em>diff</em> part of the project. I have made some assumptions and wrote some sample code to test them out, but I would like to know if you see anything I might have missed, if the assumptions are just plain wrong, or if there's something that could go wrong in a particular constelation.</p>
  
  <p><strong>Assumption 1:</strong> If files are not of equal length, they can not be the same (ie. some modification must have taken place)<br>
  <strong>Assumption 2:</strong> If two files are the same (ie. no modification has taken place) any byte sub-set of these two files will have the same hash<br>
  <strong>Assumption 3:</strong> If a byte sub-set of two files is found which does not result in the same hash, the two files are not the same (ie. have been modified)</p>
  
  <p>The code is written in Java and the hashing algorithm used is <a href="https://131002.net/blake/" rel="nofollow">BLAKE-512</a> using the <a href="https://code.google.com/p/blake-512-java-implementation/" rel="nofollow">java implementation</a> from Marc Greim.<br>
  <code>_File1</code> and <code>_File2</code> are 2 files > 1.5GB of type <code>java.io.File</code></p>
  
  <pre><code>public boolean compareStream() throws IOException {
      int i = 0;
      int step = 4096;
      boolean equal = false;
  
      FileInputStream fi1 = new FileInputStream(_File1);      
      FileInputStream fi2 = new FileInputStream(_File2);
  
      byte[] fi1Content = new byte[step];
      byte[] fi2Content = new byte[step];
  
      if(_File1.length() == _File2.length()) { //Assumption 1
          while(i*step &lt; _File1.length()) {   
  
              fi1.read(fi1Content, 0, step); //Assumption 2
              fi2.read(fi2Content, 0, step); //Assumption 2
  
              equal = BLAKE512.isEqual(fi1Content, fi2Content); //Assumption 2
  
              if(!equal) { //Assumption 3
                  break;
              }
  
              ++i;
          }
      }
  
      fi1.close();
      fi2.close();
      return equal;
  }
  </code></pre>
  
  <p>The calculation for two equal 1.5 GB files takes around 4.2 seconds. Times are of course much shorter when the files differ, especially when they are of different length since it returns immediately.</p>
  
  <p>Thank you for your suggestions :)<br>
  ..I hope this isn't too broad</p>