title: Having trouble making this assembly procedure work in Visual Studio
body: <p>I'm trying to solve this question that wants me to encrypt a message by rotating the bits of each 10 bytes of a message to the left or to the right according to a certain key, for example:</p>
  <p><code>key BYTE -2, 4, 1, 0, -3, 5, 2, -4, -4, 6</code></p>
  <p>Where the sign indicates the direction of rotation, negative being to the left, positive to the right. The numbers indicate the magnitude of rotation. So the first byte of the message will be rotated twice to the left, the second 4 times to the right and so on and we do the same with the 11th and 12th byte and so on until the end of the message.</p>
  <p>When I call this procedure, nothing happens to the message stored in memory, however:</p>
  <pre><code>EncryptR PROC USES ebx ecx esi,
  ptrkey: PTR BYTE, ; pointer to the key array
  ptrmessage: PTR BYTE, ; pointer to the plaintext message
  mlength: DWORD ; length of the message array
  ; -----------------------------------------------------------------------------
  
  mov ecx, mlength 
      inc ecx
      mov edx, [ptrkey+10]
      ; mov edi, ptrmessage
      L2:
      mov esi, ptrkey 
      dec ecx 
      cmp ecx, 0 
      je endloop
      L1:
      push ecx
      mov cl, [esi]
      cmp cl,0
      ja Skip1
      neg cl
      rol [ptrmessage], cl
      jmp Skip2
      Skip1: 
      ror [ptrmessage], cl
      Skip2: 
      pop ecx 
      inc esi 
      inc ptrmessage
      cmp [esi], edx
      je L2
      LOOP L1
  
      endloop:
      ret
  </code></pre>
  <p>I've tried using edi to point to the message but it doesn't work according to the syntax of the rotate instructions apparently. I've used ptrmessage directly for now so that I avoid build errors. I'm calling the procedure using invoke and have already declared it using the PROTO directive. I've tried doing it in a way which would reset the pointer of the key array to the first element once it is out of bounds while the message pointer keeps going.</p>
  <pre><code>MOV edx, offset Message1 
      call Strlength
      Invoke EncryptR, ADDR key, addr Message1, eax
  </code></pre>
  <p>I'm not sure what's the problem, there's the possibilty of jumping to the endloop label prematurely which would explain it but it seems obvious that that wouldn't happen the first time round. Is there an issue with the way I have been using the rotate instructions?</p>