title: RC4 implementation doesn't match openssl output
body: <p>My goal is to implement the RC4 stream cipher in C/C++, and make sure it produces the same output as when using the <code>openssl</code> command.
  Following the pseudocode on <a href="https://en.wikipedia.org/wiki/RC4" rel="nofollow noreferrer">wikipedia</a>,
  this implementation appears to work, in that it can encrypt and decrypt content.
  However, the encrypted output doesn't match the output of the equivalent <code>openssl</code> command, I'd like to understand why.</p>
  
  <pre><code>/*
      Reference: https://en.wikipedia.org/wiki/RC4
  */
  
  #include &lt;stdio.h&gt;
  #include &lt;string.h&gt;
  #include &lt;stdlib.h&gt;
  
  typedef u_int8_t byte;
  typedef struct
  {
      byte i;
      byte j;
      byte S[256];
  } Rc4State;
  
  static void swap(byte *a, byte *b)
  {
      byte temp = *a;
      *a = *b;
      *b = temp;
  }
  
  /*
      Set initial permutation.
      Initialize i and j counters for stream generation.
  */
  void rc4InitState(Rc4State *state, const byte K[256], u_int8_t klen)
  {
      byte *S = state-&gt;S;
  
      for (int i = 0; i &lt; 256; i++)
      {
          S[i] = i;
      }
  
      int j = 0;
      for (int i = 0; i &lt; 256; i++)
      {
          j = (j + S[i] + K[i % klen]) % 256;
          swap(&amp;S[i], &amp;S[j]);
      }
  
      state-&gt;i = 0;
      state-&gt;j = 0;
  }
  
  void rc4Crypt(Rc4State *state, byte buffer[], size_t len)
  {
      byte *S = state-&gt;S;
  
      for (size_t k = 0; k &lt; len; k++)
      {
          byte i = ++(state-&gt;i);
          byte j = (state-&gt;j += S[i]);
  
          swap(&amp;S[i], &amp;S[j]);
          byte t = S[i] + S[j];
  
          buffer[k] ^= S[t];
      }
  }
  
  int main(int argc, const char *argv[])
  {
      if (argc &lt; 2)
      {
          printf("usage: %s secret [-nosalt]\n", argv[0]);
          exit(1);
      }
  
      const char *key = argv[1];
      int klen = strlen(key);
  
      if (!(1 &lt;= klen &amp;&amp; klen &lt; 256))
      {
          printf("secret length must be between 1 and 255 chars, got %d\n", klen);
          exit(1);
      }
  
      byte *K = new byte[256];
      for (int i = 0; i &lt; klen; i++)
      {
          K[i] = key[i];
      }
  
      Rc4State *state = (Rc4State *)malloc(sizeof(Rc4State));
      rc4InitState(state, K, klen);
  
      byte *buffer = new byte[256];
      while (1)
      {
          size_t size = fread(buffer, sizeof(byte), 256, stdin);
          if (ferror(stdin))
              break;
  
          rc4Crypt(state, buffer, size);
          fwrite(buffer, sizeof(byte), size, stdout);
  
          if (feof(stdin))
              break;
      }
  }
  </code></pre>
  
  <p>I compile it with:</p>
  
  <pre><code>g++ prog.cpp
  </code></pre>
  
  <p>Where <code>g++ --version</code> is:</p>
  
  <pre><code>$ g++ --version
  Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/c++/4.2.1
  Apple clang version 11.0.3 (clang-1103.0.32.29)
  Target: x86_64-apple-darwin19.4.0
  Thread model: posix
  InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
  </code></pre>
  
  <p>I test it with:</p>
  
  <pre><code>path=path/to/some/file
  ./a.out foo &lt; "$path" &gt; /tmp/a
  openssl rc4 -pass pass:foo -nosalt &lt; "$path" &gt; /tmp/b
  cmp /tmp/a /tmp/b
  </code></pre>
  
  <p>Unfortunately the outputs don't match, I'd like to understand why.</p>
  
  <p>In case it helps, consider this concrete example:</p>
  
  <pre><code>for i in {1..100}; do echo $i; done &gt; /tmp/sample.txt
  </code></pre>
  
  <p>Head of <code>hexdump</code> using my implementation:</p>
  
  <pre><code>$ ./a.out foo &lt; /tmp/sample.txt | hexdump -C | head
  00000000  9a 28 df 23 fb 6b 6f 38  03 80 83 ff aa a4 6b 81  |.(.#.ko8......k.|
  00000010  5d 28 e8 a7 bf b1 4d d0  4b 34 3c 65 7c 21 f9 26  |](....M.K4&lt;e|!.&amp;|
  00000020  d0 aa 2f 75 e6 96 9c d3  df 64 54 3b 6e 5b cc 47  |../u.....dT;n[.G|
  00000030  50 76 23 48 f6 88 f8 c7  88 47 f5 89 4f 3e 01 5b  |Pv#H.....G..O&gt;.[|
  00000040  e1 b4 f9 03 f3 56 48 9c  c2 a1 45 dc a1 ed da ce  |.....VH...E.....|
  00000050  99 1e d2 ab 65 29 d8 8f  49 a3 bf 88 7c 49 d2 9a  |....e)..I...|I..|
  00000060  78 f7 ed 04 ec 23 f4 8a  18 06 7d ec 74 90 12 60  |x....#....}.t..`|
  00000070  94 f9 a5 9b f8 97 c4 9b  31 94 eb dd 32 66 5e 8a  |........1...2f^.|
  00000080  03 4d c1 d1 75 b5 89 9b  19 1f 6f 55 39 59 97 78  |.M..u.....oU9Y.x|
  00000090  c6 64 81 85 8e 9c b8 0f  ef 29 90 77 29 02 0e 93  |.d.......).w)...|
  </code></pre>
  
  <p>Head of <code>hexdump</code> using <code>openssl</code> (on OSX, version <code>OpenSSL 1.0.1e 11 Feb 2013</code>):</p>
  
  <pre><code>$ openssl rc4 -pass pass:foo -nosalt &lt; /tmp/sample.txt | hexdump -C | head
  00000000  de c0 1e 94 70 ef f2 55  45 69 f0 c9 71 94 30 32  |....p..UEi..q.02|
  00000010  b4 6e fd d5 43 ef 4c 56  a0 58 00 8e f2 33 84 cd  |.n..C.LV.X...3..|
  00000020  e2 d4 14 3b 78 7c 27 34  1a f2 2c e5 3a c2 9a 6e  |...;x|'4..,.:..n|
  00000030  ab 20 e5 30 84 4f 17 b5  1a 2f 76 f6 b2 30 48 81  |. .0.O.../v..0H.|
  00000040  39 70 50 21 f2 fc dc 0b  11 eb 0e e8 fa 0f ab 7c  |9pP!...........||
  00000050  02 28 0a 0e 06 8e f6 44  2b 0d 67 c0 88 12 a7 74  |.(.....D+.g....t|
  00000060  66 a3 18 b5 f8 ea d4 7b  05 84 cf 56 42 07 0c 8c  |f......{...VB...|
  00000070  d9 8a c3 fc dc 30 9a ef  4c ca 00 b8 7d 15 32 ac  |.....0..L...}.2.|
  00000080  7d 3e 54 19 d8 b6 a6 22  d1 14 a1 a4 12 b0 a5 aa  |}&gt;T...."........|
  00000090  63 e1 63 87 8b a1 58 88  6c 19 3d 32 09 07 0f bf  |c.c...X.l.=2....|
  </code></pre>
  
  <p>What am I missing?</p>
  
  <p>Interestingly, the same <code>openssl</code> command on the same sample gives a different result on a Debian system (version <code>OpenSSL 1.1.1c  28 May 2019</code>):</p>
  
  <pre><code>$ openssl rc4 -pass pass:foo -nosalt &lt; /tmp/sample.txt | hexdump -C | head
  *** WARNING : deprecated key derivation used.
  Using -iter or -pbkdf2 would be better.
  00000000  13 b9 ff 0f bf 7e e7 4b  6b 6f 28 86 10 a5 a0 cd  |.....~.Kko(.....|
  00000010  86 6d 6b d5 58 8e c1 d0  65 3e cd ae b0 c5 64 5f  |.mk.X...e&gt;....d_|
  00000020  f8 fc 93 76 1e ce c2 d3  8e 1a e1 7d 78 12 ee 3a  |...v.......}x..:|
  00000030  b1 a8 43 ec c4 fb 06 ed  f9 3a fd 8d c7 d9 18 4d  |..C......:.....M|
  00000040  e5 a4 6a 9e 59 f4 a6 37  b3 0d 6f c8 cb a4 fb 6a  |..j.Y..7..o....j|
  00000050  31 8a 3a 5e f3 df 41 6f  d2 3d 53 aa ee 6f cb 31  |1.:^..Ao.=S..o.1|
  00000060  8b 43 e8 f5 45 91 46 4c  15 ab d1 0e 5d 6a 19 90  |.C..E.FL....]j..|
  00000070  c7 fc f2 10 89 e3 bb 1d  33 d7 9c 42 70 31 bf 05  |........3..Bp1..|
  00000080  e1 dc 91 47 92 a9 d9 da  de c3 a1 b3 20 5d c7 d5  |...G........ ]..|
  00000090  d1 6f 8e 57 05 f6 6e 87  38 49 bc d8 90 29 9a 4d  |.o.W..n.8I...).M|
  </code></pre>
  
  <p>So perhaps the <code>openssl</code> implementation/version is important?</p>