title: MPI and Variable overwrite?
body: <p>I try to run a distribuded application where is used to calculate a <a href="https://en.bmstu.wiki/Burmester-Desmedt_(protocol)" rel="nofollow noreferrer">burmester desmedt</a> key agreement.</p>
  
  <p>The code that does this is the following:</p>
  
  <pre><code>#include "mpi.h"
  #include &lt;stdio.h&gt;
  
  // Openssl
  #include &lt;openssl/conf.h&gt;
  #include &lt;openssl/evp.h&gt;
  #include &lt;openssl/err.h&gt;
  
  // Custom
  #include "message.h"
  #include "dh.h"
  
  void cleanup(DH *secret);
  
  void cleanup(DH *secret) {
    EVP_cleanup();
    CRYPTO_cleanup_all_ex_data();
    OPENSSL_free(secret);
    ERR_free_strings();
  }
  
  int main(int argc, char *argv[]) {
    int rank, size;
  
  
    MPI_Init( &amp;argc, &amp;argv );
    MPI_Comm_rank( MPI_COMM_WORLD, &amp;rank );
    MPI_Comm_size( MPI_COMM_WORLD, &amp;size );
  
    /* Load the human readable error strings for libcrypto */
    ERR_load_crypto_strings();
    /* Load all digest and cipher algorithms */
    OpenSSL_add_all_algorithms();
    /* Load config file, and other important initialisation */
    OPENSSL_config(NULL);
  
    DH *secret;
  
    printf("RANK %d, Generating Diffie Hellman Keys\n", rank);
    fflush(stdout);
    if(-1 == generateKeys(secret)) {
      cleanup(secret);
      fprintf(stderr, "RANK %d, Failed to intialize key\n",rank);
      fflush(stderr);
      return -1;
    }
  
    printf("RANK %d, Keys generated\n",rank);
    fflush(stdout);
  
    // Set up Barrier for cpommunications
    MPI_Barrier(MPI_COMM_WORLD);
  
    printf("RANK %d, Publishing Keys\n", rank);
    fflush(stdout);
  
    if(secret == NULL) {
      cleanup(secret);
      fprintf(stderr, "RANK %d, Error on Generating the Diffie Hellman\n",rank);
      fflush(stderr);
      return -1;
    }
  
    if(MPIbcastBigNum(secret-&gt;pub_key, rank, "Publishing Public Key") == -1){
     cleanup(secret);
     return -1;
    }
  
    /*Cleanup */
    cleanup(secret);
    MPI_Finalize();
    return 0;
  }
  </code></pre>
  
  <p>And there are helper methods for initializing the DH and for the messaging (in order to save space I post the code via links):</p>
  
  <ul>
  <li>Diffie Hellman: <a href="https://github.com/pc-magas/burmester-desmedt/blob/master/dh.c" rel="nofollow noreferrer">https://github.com/pc-magas/burmester-desmedt/blob/master/dh.c</a></li>
  <li>Messaging: <a href="https://github.com/pc-magas/burmester-desmedt/blob/master/message.c" rel="nofollow noreferrer">https://github.com/pc-magas/burmester-desmedt/blob/master/message.c</a></li>
  </ul>
  
  <p>Also I compile it via this <a href="https://github.com/pc-magas/burmester-desmedt/blob/master/Makefile" rel="nofollow noreferrer">Makefile</a>. So I want to know why I the following execution turns into segfault?</p>
  
  <pre><code>mpirun -np 3 ./builds/main
  RANK 1, Generating Diffie Hellman Keys
  RANK 2, Generating Diffie Hellman Keys
  RANK 0, Generating Diffie Hellman Keys
  RANK 1, Keys generated
  RANK 2, Keys generated
  RANK 0, Keys generated
  RANK 2, Publishing Keys
  [pcmagas-System-Product-Name:19304] *** Process received signal ***
  [pcmagas-System-Product-Name:19304] Signal: Segmentation fault (11)
  [pcmagas-System-Product-Name:19304] Signal code:  (128)
  [pcmagas-System-Product-Name:19304] Failing at address: (nil)
  [pcmagas-System-Product-Name:19304] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x11390)[0x7f485819c390]
  [pcmagas-System-Product-Name:19304] [ 1] /lib/x86_64-linux-gnu/libcrypto.so.1.0.0(BN_num_bits+0x1d)[0x7f485873acfd]
  [pcmagas-System-Product-Name:19304] [ 2] ./builds/main[0x401461]
  [pcmagas-System-Product-Name:19304] [ 3] ./builds/main[0x40127b]
  [pcmagas-System-Product-Name:19304] [ 4] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7f4857de1830]
  [pcmagas-System-Product-Name:19304] [ 5] ./builds/main[0x400ff9]
  [pcmagas-System-Product-Name:19304] *** End of error message ***
  RANK 1, Publishing Keys
  [pcmagas-System-Product-Name:19303] *** Process received signal ***
  [pcmagas-System-Product-Name:19303] Signal: Segmentation fault (11)
  [pcmagas-System-Product-Name:19303] Signal code:  (128)
  [pcmagas-System-Product-Name:19303] Failing at address: (nil)
  [pcmagas-System-Product-Name:19303] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x11390)[0x7f2e56b90390]
  [pcmagas-System-Product-Name:19303] [ 1] /lib/x86_64-linux-gnu/libcrypto.so.1.0.0(BN_num_bits+0x1d)[0x7f2e5712ecfd]
  [pcmagas-System-Product-Name:19303] [ 2] ./builds/main[0x401461]
  [pcmagas-System-Product-Name:19303] [ 3] ./builds/main[0x40127b]
  [pcmagas-System-Product-Name:19303] [ 4] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7f2e567d5830]
  [pcmagas-System-Product-Name:19303] [ 5] ./builds/main[0x400ff9]
  [pcmagas-System-Product-Name:19303] *** End of error message ***
  RANK 0, Publishing Keys
  [pcmagas-System-Product-Name:19302] *** Process received signal ***
  [pcmagas-System-Product-Name:19302] Signal: Segmentation fault (11)
  [pcmagas-System-Product-Name:19302] Signal code:  (128)
  [pcmagas-System-Product-Name:19302] Failing at address: (nil)
  [pcmagas-System-Product-Name:19302] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x11390)[0x7fd967a55390]
  [pcmagas-System-Product-Name:19302] [ 1] /lib/x86_64-linux-gnu/libcrypto.so.1.0.0(BN_num_bits+0x1d)[0x7fd967ff3cfd]
  [pcmagas-System-Product-Name:19302] [ 2] ./builds/main[0x401461]
  [pcmagas-System-Product-Name:19302] [ 3] ./builds/main[0x40127b]
  [pcmagas-System-Product-Name:19302] [ 4] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7fd96769a830]
  [pcmagas-System-Product-Name:19302] [ 5] ./builds/main[0x400ff9]
  [pcmagas-System-Product-Name:19302] *** End of error message ***
  --------------------------------------------------------------------------
  mpirun noticed that process rank 2 with PID 19304 on node pcmagas-System-Product-Name exited on signal 11 (Segmentation fault).
  --------------------------------------------------------------------------
  Makefile:16: recipe for target 'run' failed
  make: *** [run] Error 139
  </code></pre>
  
  <p>Is the <code>DH</code> Object somehow overwritten (as fas as I know OpenSSL is not thread and process safe and I attempt to run it via separate processes).</p>