WebRTC DTLS-SRTP OpenSSL Server Handshake Failure <p><strong><em>Here is my procedure in OpenSSL Server Mode,</em></strong></p> <p><strong>Initialization Part of SSL and BIO variables:</strong> </p> <pre><code>map&lt;int, SSL&gt; m_SSLMap; map&lt;int, BIO&gt; m_BioWriteMap; map&lt;int, BIO&gt; m_BioReadMap; int InitializeServerNegotiationMode(int iFd) { SSL *pServSslFd; BIO *pWb, *pRb; pServSslFd = SSL_new(m_pCtx); assert(pServSslFd); if ( SSL_version(pServSslFd) == DTLS1_VERSION) { pWb = BIO_new(BIO_s_mem()); pRb = BIO_new(BIO_s_mem()); assert(pWb); assert(pRb); SSL_set_bio(pServSslFd, pRb, pWb); SSL_set_accept_state(pServSslFd); } m_SSLMap[iFd] = *pServSslFd; m_BioReadMap[iFd] = *pRb; m_BioWriteMap[iFd] = *pWb; return INITIALIZATION_SUCCESS; } </code></pre> <p><strong>Server Mode Negotiation Operations when DTLS data comes to the server:</strong></p> <pre><code>int ServerModeDTLSNegotiation(int iChannel, const char *pBuff, const int iLen, int iFd) { SSL *pServSslFd; BIO *pRbio; BIO *pWbio; pServSslFd = &amp;m_SSLMap[iFd]; pRbio = &amp;m_BioReadMap[iFd]; pWbio = &amp;m_BioWriteMap[iFd]; char buff[4096]; memset(buff, 0, strlen(buff)); BIO_write(pRbio, pBuff, iLen); if(!SSL_is_init_finished(pServSslFd)) { int iRet = SSL_do_handshake(pServSslFd); } int iNewLen = BIO_read(pWbio, buff, 2048); if(iNewLen&gt;0) { char *pNewData = new char[iNewLen+1]; for(int i=0;i&lt;iNewLen;i++) pNewData[i] = buff[i]; m_pEventHandler-&gt;SendReply(iChannel, (unsigned char *)pNewData, iNewLen); } else { printf("[DTLS]:: HandShaking Response failed for this data, return -1; } return NEGOTIATION_SUCCESS; } </code></pre> <p>Here I am attaching Wireshark TCP-Dump for better monitoring about the issue. </p> <p><a href="https://www.dropbox.com/s/quidcs6gilnvt2o/WebRTC%20DTLS%20Handshake%20Failure.pcapng?dl=0">https://www.dropbox.com/s/quidcs6gilnvt2o/WebRTC%20DTLS%20Handshake%20Failure.pcapng?dl=0</a></p> <p>Now, I am confident about my initialization of SSL_CTX variable. Because, Sometimes Handshake successfully negotiate for every port. But sometimes Handshake fails for one or two port. I am working for 5 days to solve WebRTC DTLS Server Mode Negotiation for Google Chrome. But I haven't found the root cause for this problem. </p>