title: base64 digest- invalid input?
body: <p>I have here the following data for which I have to find the sha1 digest using openssl.</p>
  
  <p>data:</p>
  
  <pre><code>AwAIAOwIAAABABwAgAIAABYAAAAAAAAAAAAAAHQAAAAAAAAAAAAAAAgAAAAkAAAAQgAAAFQAAABsAAAAhgAAAJgAAACuAAAAwgAAAM4AAADsAAAAAgEAAAwBAAAoAQAARgEAAFgBAACwAQAAtAEAANABAADkAQAA+gEAAAIAaQBkAAAADABsAGEAeQBvAHUAdABfAHcAaQBkAHQAaAAAAA0AbABhAHkAbwB1AHQAXwBoAGUAaQBnAGgAdAAAAAcAZwByAGEAdgBpAHQAeQAAAAoAYgBhAGMAawBnAHIAbwB1AG4AZAAAAAsAbwByAGkAZQBuAHQAYQB0AGkAbwBuAAAABwBwAGEAZABkAGkAbgBnAAAACQB0AGUAeAB0AEMAbwBsAG8AcgAAAAgAdABlAHgAdABTAGkAegBlAAAABAB0AGUAeAB0AAAADQBwAGEAZABkAGkAbgBnAEIAbwB0AHQAbwBtAAAACQBzAGMAYQBsAGUAVAB5AHAAZQAAAAMAcwByAGMAAAAMAHAAYQBkAGQAaQBuAGcAUgBpAGcAaAB0AAAADQBsAGEAeQBvAHUAdABfAHcAZQBpAGcAaAB0AAAABwBhAG4AZAByAG8AaQBkAAAAKgBoAHQAdABwADoALwAvAHMAYwBoAGUAbQBhAHMALgBhAG4AZAByAG8AaQBkAC4AYwBvAG0ALwBhAHAAawAvAHIAZQBzAC8AYQBuAGQAcgBvAGkAZAAAAAAAAAAMAEwAaQBuAGUAYQByAEwAYQB5AG8AdQB0AAAACABUAGUAeAB0AFYAaQBlAHcAAAAJAEkAbQBhAGcAZQBWAGkAZQB3AAAABgBCAHUAdAB0AG8AbgAAAAAAgAEIAEQAAADQAAEB9AABAfUAAQGvAAEB1AABAcQAAQHVAAEBmAABAZUAAQFPAQEB2QABAR0BAQEZAQEB2AABAYEBAQEAARAAGAAAABEAAAD/////DwAAABAAAAACARAAsAAAABEAAAD//////////xIAAAAUABQABwAAAAAAAAAQAAAAAwAAAP////8IAAAREQAAABAAAAAFAAAA/////wgAABABAAAAEAAAAAAAAAD/////CAAAAR0AB38QAAAABAAAAP////8IAAABEQAGfxAAAAAGAAAA/////wgAAAUBEAAAEAAAAAEAAAD/////CAAAEP////8QAAAAAgAAAP////8IAAAQ/////wIBEACcAAAAGgAAAP//////////EwAAABQAFAAGAAAAAAAAABAAAAAIAAAA/////wgAAAUCEgAAEAAAAAcAAAD/////CAAAARAABn8QAAAACgAAAP////8IAAAFARgAABAAAAABAAAA/////wgAABD/////EAAAAAIAAAD/////CAAAEP7///8QAAAACQAAAP////8IAAABRwAIfwMBEAAYAAAAIAAAAP//////////EwAAAAIBEAB0AAAAIgAAAP//////////EgAAABQAFAAEAAAAAAAAABAAAAAFAAAA/////wgAABAAAAAAEAAAAAQAAAD/////CAAAAREABn8QAAAAAQAAAP////8IAAAQ/////xAAAAACAAAA/////wgAABD+////AgEQAIgAAAAoAAAA//////////8UAAAAFAAUAAUAAAAAAAAAEAAAAA0AAAD/////CAAABQEYAAAQAAAAAQAAAP////8IAAAQ/v///xAAAAACAAAA/////wgAABD+////EAAAAAwAAAD/////CAAAAQEAAn8QAAAACwAAAP////8IAAAQBQAAAAMBEAAYAAAALQAAAP//////////FAAAAAIBEAB0AAAALwAAAP//////////EgAAABQAFAAEAAAAAAAAABAAAAAFAAAA/////wgAABABAAAAEAAAAAEAAAD/////CAAABQEAAAAQAAAAAgAAAP////8IAAAQ/v///xAAAAAOAAAA/////wgAAAQAAIA/AgEQAHQAAAA1AAAA//////////8VAAAAFAAUAAQAAAAAAAAAEAAAAAAAAAD/////CAAAASgAB38QAAAAAQAAAP////8IAAAQ/////xAAAAACAAAA/////wgAABD+////EAAAAAkAAAD/////CAAAARUACH8DARAAGAAAADgAAAD//////////xUAAAACARAAdAAAADoAAAD//////////xUAAAAUABQABAAAAAAAAAAQAAAAAAAAAP////8IAAABKgAHfxAAAAABAAAA/////wgAABD/////EAAAAAIAAAD/////CAAAEP7///8QAAAACQAAAP////8IAAABGgAIfwMBEAAYAAAAPQAAAP//////////FQAAAAMBEAAYAAAAPwAAAP//////////EgAAAAIBEAB0AAAAQQAAAP//////////EgAAABQAFAAEAAAAAAAAABAAAAAFAAAA/////wgAABABAAAAEAAAAAEAAAD/////CAAABQEAAAAQAAAAAgAAAP////8IAAAQ/v///xAAAAAOAAAA/////wgAAAQAAIA/AgEQAHQAAABHAAAA//////////8VAAAAFAAUAAQAAAAAAAAAEAAAAAAAAAD/////CAAAASkAB38QAAAAAQAAAP////8IAAAQ/////xAAAAACAAAA/////wgAABD+////EAAAAAkAAAD/////CAAAARYACH8DARAAGAAAAEoAAAD//////////xUAAAACARAAdAAAAEwAAAD//////////xUAAAAUABQABAAAAAAAAAAQAAAAAAAAAP////8IAAABKwAHfxAAAAABAAAA/////wgAABD/////EAAAAAIAAAD/////CAAAEP7///8QAAAACQAAAP////8IAAABGQAIfwMBEAAYAAAATwAAAP//////////FQAAAAMBEAAYAAAAUQAAAP//////////EgAAAAMBEAAYAAAAUwAAAP//////////EgAAAAMBEAAYAAAAVQAAAP//////////EgAAAAEBEAAYAAAAVQAAAP////8PAAAAEAAAABgAAAA9AAAA//////////8fAAAAAgEQAGAAAAA/AAAA//////////8eAAAAFAAUAAMAAAAAAAAAGQAAAAUAAAD/////CAAAEAAAAAAZAAAAAAAAAP////8IAAAQ/v///xkAAAABAAAA/////wgAABD+////AgEQAMQAAABEAAAA//////////8gAAAAFAAUAAgAAAAAAAAAGQAAABIAAAD/////CAAABQIOAAAZAAAAEQAAAP////8IAAARAQAAABkAAAAQAAAA/////wgAAAEGAAZ/GQAAAAIAAAD/////CAAAARIAB38ZAAAAEwAAAP////8IAAAFAQQAABkAAAAAAAAA/////wgAABD+////GQAAAAEAAAD/////CAAAEP7///8ZAAAADwAAAP////8IAAABMwAIfwMBEAAYAAAASwAAAP//////////IAAAAAIBEACIAAAATQAAAP//////////IAAAABQAFAAFAAAAAAAAABkAAAASAAAA/////wgAAAUCDgAAGQAAABAAAAD/////CAAAAQYABn8ZAAAAAgAAAP////8IAAABEwAHfxkAAAAAAAAA/////wgAABD+////GQAAAAEAAAD/////CAAAEP7///8DARAAGAAAAFEAAAD//////////yAAAAADARAAGAAAAFMAAAD//////////x4AAAACARAAYAAAAFUAAAD//////////x4AAAAUABQAAwAAAAAAAAAZAAAABQAAAP////8IAAAQAAAAABkAAAAAAAAA/////wgAABD+////GQAAAAEAAAD/////CAAAEP7///8CARAAxAAAAFoAAAD//////////yAAAAAUABQACAAAAAAAAAAZAAAAEgAAAP////8IAAAFAg4AABkAAAARAAAA/////wgAABEBAAAAGQAAABAAAAD/////CAAAAQYABn8ZAAAAAgAAAP////8IAAABFAAHfxkAAAATAAAA/////wgAAA
  </code></pre>
  
  <p>The digest as given to me is:
  Wk2pJnOErEHsElMw4TMX+rjHsQQ=</p>
  
  <p>But when I use(f1= file where I copied the above data):</p>
  
  <p>base64 -d f1.txt | openssl dgst -sha1 -binary | base64</p>
  
  <p>I get a "base64: invalid input" error and the following digest which seems completely different :(</p>
  
  <p>BaRlDid73RYBFMgqveC8G+gFBBU=</p>
  
  <p>Can somebody confirm and explain if there is some mistake??</p>
  
  <p>UPDATED:</p>
  
  <p>Scenario: Client's binary file is base64 encoded and sent to server. Server decodes this and computes the sha1 digest. Since I have client's base64 encoded sha1 digest, the server also encodes the digest to base64. Now these two should match. And it doesn't! I receive all data. I have rechecked it. I shall present part of the code here:</p>
  
  <pre><code>//RCVBUFSIZE = 1024 (defined)
  void HandleClient(int clntSocket)
  {
  char echoBuffer[RCVBUFSIZE] ;        /* Buffer for echo string */
      memset(echoBuffer, 0, RCVBUFSIZE);
      char inBuffer;        /* Buffer for first string */
      char recv_data;
      int recvMsgSize = 0;                    /* Size of received message */
      char replyBuffer[32];
      int bytes_received = 0;
      int rv = 0;
      int connected = clntSocket;
      int len= 0;
      int i = 0;
       EVP_MD_CTX md_ctx;
       const EVP_MD *md;
       unsigned char md_value[EVP_MAX_MD_SIZE];
       unsigned int md_len;
       OpenSSL_add_all_digests();
       md = EVP_get_digestbyname("sha1");
       EVP_MD_CTX_init(&amp;md_ctx);
       EVP_DigestInit_ex(&amp;md_ctx, md, NULL);
  
      /* Receive message from client */
      while (((bytes_received = recv(connected,&amp;inBuffer,1,0)) &gt; 0) &amp;&amp; (inBuffer != '\n')){
  
      /* Send received string and receive again until end of transmission */
      if (bytes_received  &gt; 0)      /* zero indicates end of transmission */
      {
      printf("Message received from Client is : %c\n", inBuffer);
      char n = inBuffer;
      int indicator =  0;
      int current = 0;
      unsigned long fileLen;
  
      if(n =='6'){
          if ((recvMsgSize = recv(connected, echoBuffer, RCVBUFSIZE, 0)) &lt; 0)
          DieWithError("recv() failed");
          printf("no. of bytes got : %d\n", recvMsgSize);
          if (recvMsgSize &gt; 0)
          echoBuffer[recvMsgSize] = '\0';
          len= atoi(echoBuffer);
          char *data =NULL;
          printf("length of following message : %d\n", len);
          if(len&gt;0){
      for( i = RCVBUFSIZE; i &lt; (len+RCVBUFSIZE); i=i+RCVBUFSIZE){
                  if(i&gt;len)
                  recvMsgSize = recv(connected, echoBuffer, (len - (i-RCVBUFSIZE)), 0);
                  else
                  recvMsgSize = recv(connected, echoBuffer, RCVBUFSIZE, 0);               
                  echoBuffer[recvMsgSize] = '\0';
                  decode(echoBuffer, recvMsgSize, "file_out");
                  data = readFileBuffer("file_out");
                  EVP_DigestUpdate(&amp;md_ctx, data, strlen(data));
              }
          }
          len = 0;
          memset(echoBuffer, 0, RCVBUFSIZE);
          recvMsgSize = 0;
  
      }
  
  
      if (n =='5'){
              printf("Update Digest Over- Calculate Final Dgst!!!!! \n");
              n= 0;
  
              EVP_DigestFinal_ex(&amp;md_ctx, md_value, &amp;md_len); //retrieve digest from ctx unto md_value and #bytes written is copied into   md_len
              EVP_MD_CTX_cleanup(&amp;md_ctx);
              FILE *f;
               f = fopen("file_sha1", "w");
              printf("\n");
              printf("******************************************************\n ");
              printf("Digest is: ");
              for(i = 0; i &lt; md_len; i++){
                  if ( f !=NULL){
                      fputc(md_value[i], f);
                      }
              printf("%02x", md_value[i]);
              }
              printf("\n");
              printf("******************************************************\n ");
              fclose(f);
      }
  printf("socket closing\n");
      close(connected);    /* Close client socket */
  }
  }
  
  char *readFileBuffer(char *name)
  {
      FILE *file;
      char *buffer = NULL;
      unsigned long fileLen;
      //Open file
      file = fopen(name, "rb");
      if (!file)
      {
          fprintf(stderr, "Unable to open file %s", name);
          return;
      }
      //Get file length
      fseek(file, 0, SEEK_END);
      fileLen=ftell(file);
      printf("file length  = %ld\n", fileLen);
      fseek(file, 0, SEEK_SET);
  
      //printf("Allocate memory\n");
      buffer=(char *)malloc(fileLen+1);
      printf("length of write buffer = %d\n", strlen(buffer));
      if (!buffer)
      {
          fprintf(stderr, "Memory error!");
  
      }
  
      long int n = fread(buffer,1, fileLen,file);
      buffer[n] = '\0';
      printf("Read no. of bytes = %ld into buffer \n", n);
      printf("len of buffer  %d  \n", strlen(buffer));
      if (!buffer)
      {
          fprintf(stderr, "Memory error!");
          fclose(file);
      }
      fclose(file);
      //free(name);
      return buffer;
  
  }
  
  // reads b64 encoded msg (ReadBuffer) and writes to WriiteFile.
  void decode(char *ReadBuffer, int Length, char *WriteFile)
  {
      char *msg = (char *)malloc(Length);
      memset(msg, 0x00, Length);
      int readbytes = -1;
  
      printf("buffer write file %s\n", WriteFile);
  
      // the decode msg is written to this bio
      BIO *fileWrBIO = BIO_new_file(WriteFile, "w");
  
      BIO *b64 = BIO_new(BIO_f_base64());
      BIO *bio = BIO_new_mem_buf(ReadBuffer, Length);
      bio = BIO_push(b64, bio);
      BIO_set_flags(bio,BIO_FLAGS_BASE64_NO_NL);
  
      while ((readbytes = BIO_read(bio, msg, Length)) &gt; 0)
      {
      printf("readbytes:  %d\n", readbytes);
         BIO_write(fileWrBIO, msg, readbytes);
         BIO_flush(fileWrBIO);
         memset(msg, 0x00, sizeof(msg));
      }
      free(msg);
      BIO_free_all(bio);
      BIO_free_all(fileWrBIO);
  }
  </code></pre>