{"action":"created","comment":{"url":"https://api.github.com/repos/VoltDB/voltdb/pulls/comments/84379292","pull_request_review_id":4933750,"id":84379292,"diff_hunk":"@@ -1489,4 +1544,51 @@ private static void openURL()\n             // ignore any error\n         }\n     }\n+\n+    /**\n+     * Creates the key managers required to initiate the {@link SSLContext}, using a JKS keystore as an input.\n+     *\n+     * @param filepath - the path to the JKS keystore.\n+     * @param keystorePassword - the keystore's password.\n+     * @param keyPassword - the key's passsword.\n+     * @return {@link KeyManager} array that will be used to initiate the {@link SSLContext}.\n+     * @throws Exception\n+     */\n+    protected static KeyManager[] createKeyManagers(String filepath, String keystorePassword, String keyPassword) throws Exception {","path":"src/frontend/org/voltdb/utils/SQLCommand.java","position":132,"original_position":132,"commit_id":"51f9e4a69eee716ce355a4b5b5857e7f8336969a","original_commit_id":"51f9e4a69eee716ce355a4b5b5857e7f8336969a","user":{"login":"vtkstef","id":662716,"avatar_url":"https://avatars.githubusercontent.com/u/662716?v=3","gravatar_id":"","url":"https://api.github.com/users/vtkstef","html_url":"https://github.com/vtkstef","followers_url":"https://api.github.com/users/vtkstef/followers","following_url":"https://api.github.com/users/vtkstef/following{/other_user}","gists_url":"https://api.github.com/users/vtkstef/gists{/gist_id}","starred_url":"https://api.github.com/users/vtkstef/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vtkstef/subscriptions","organizations_url":"https://api.github.com/users/vtkstef/orgs","repos_url":"https://api.github.com/users/vtkstef/repos","events_url":"https://api.github.com/users/vtkstef/events{/privacy}","received_events_url":"https://api.github.com/users/vtkstef/received_events","type":"User","site_admin":false},"body":"procedure throws a generic Exception. If you want a generic exception create a descriptive runtime exception like TLSSetupException and use it to wrap check exceptions btw log them too","created_at":"2016-10-20T21:26:27Z","updated_at":"2016-10-20T21:26:58Z","html_url":"https://github.com/VoltDB/voltdb/pull/3957#discussion_r84379292","pull_request_url":"https://api.github.com/repos/VoltDB/voltdb/pulls/3957","_links":{"self":{"href":"https://api.github.com/repos/VoltDB/voltdb/pulls/comments/84379292"},"html":{"href":"https://github.com/VoltDB/voltdb/pull/3957#discussion_r84379292"},"pull_request":{"href":"https://api.github.com/repos/VoltDB/voltdb/pulls/3957"}}},"pull_request":{"url":"https://api.github.com/repos/VoltDB/voltdb/pulls/3957","id":87795196,"html_url":"https://github.com/VoltDB/voltdb/pull/3957","diff_url":"https://github.com/VoltDB/voltdb/pull/3957.diff","patch_url":"https://github.com/VoltDB/voltdb/pull/3957.patch","issue_url":"https://api.github.com/repos/VoltDB/voltdb/issues/3957","number":3957,"state":"open","locked":false,"title":"Eng 10878","user":{"login":"miketex","id":13450381,"avatar_url":"https://avatars.githubusercontent.com/u/13450381?v=3","gravatar_id":"","url":"https://api.github.com/users/miketex","html_url":"https://github.com/miketex","followers_url":"https://api.github.com/users/miketex/followers","following_url":"https://api.github.com/users/miketex/following{/other_user}","gists_url":"https://api.github.com/users/miketex/gists{/gist_id}","starred_url":"https://api.github.com/users/miketex/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/miketex/subscriptions","organizations_url":"https://api.github.com/users/miketex/orgs","repos_url":"https://api.github.com/users/miketex/repos","events_url":"https://api.github.com/users/miketex/events{/privacy}","received_events_url":"https://api.github.com/users/miketex/received_events","type":"User","site_admin":false},"body":"We're trying to do SSL with non blocking io, with NIO.  This is accomplished using SSLEngine.\r\n\r\nSSLEngine supports encryption and decryption of individual messages, it works independently of the transport layer.  SSLEngine is used to encrypt and decrypt messages that are passed between two peers, it doesn't have a sense of client or server.\r\n\r\nBecause SSLEngine is transport independent, it doesn't touch the SocketChannel being used to read and write messages.  So our code reads from the socket, and if ssl is configured for the port involved then our code uses SSLEngine to decrypt the message once it's been read.  Similarly on writing, if ssl is configured for a port then SSLEngine is used to encrypt the message before it's written to the socket; our code actually writes the message onto the socket.\r\n\r\nWhen an SSL connection is established, the two parties involved go through a handshaking process.  So if sqlcmd connects to the server, both sqlcmd and the server need to agree that the ssl is enabled; they then go through the handshake.\r\n\r\nA number of messages are passed back and forth during a handshake, but SSLEngine encrypts and decrypts only individual messages.  So our code needs to manage the handshake process, this code is in a class org.voltdb.client.SSLHandshaker.  After the connection is established, an SSLEngine that's being used to encrypt or decrypt messages needs to have participated in the handshake.  So an SSLEngine holds state as a result of handshaking, it needs that state to know how to encrypt and decrypt messages.\r\n\r\nSo the task is to agree, on the client and server, that ssl is enabled and then to handshake at the right point in time as the connection is established.  The SSLEngine that was used to handshake then needs to be plumbed into the code that will read and write over the encrypted port that was established.  So the client and server will then encrypt and decrypt all the messages that they pass back and forth over the port.\r\n\r\nIn our case, two things make this a little more complicated.  One is that we want to know the serialized size of a message when we read it and write it, but in the case of ssl we won't know the size of a message until it's been encrypted.  But we don't want to do the encryption up front, we want to defer it so that it's not done in the network thread.  \r\n\r\nA second complication is that TLS has a max message size of 16k.  This means that SSLEngine will encrypt/decrypt a buffer of this size or less.  When we have a message bigger than this, we need to break it into chunks, encrypt and send each chunk, and then assemble the chunks on the receiving end.  So a given message can consist of some number of chunks.\r\n\r\nThis was implemented without changing our wire protocol.  Each chunk consists of a four bytes size that's sent in clear.  This size is the size of the message chunk that follows, and the chunk that follows is encrypted.  For each chunk, we read it's size, in clear, read that many bytes into a buffer, and decrypt the buffer.\r\n\r\nWhen the first chunk is read, and the encrypted part is decrypted, the first four decrypted bytes rerpresent the full size of the decrypted message.  We'll continue to decrypt chunks until we reach that size.  Necessarily the last chunk needs to decrypt exactly to that size or there's a bug.  That is, when we decrypt the last chunk and add it the bytes we previously decrypted, the full decrypted message needs to be the expected size.\r\n\r\nThe same scheme is used even when a message consists of only one chunk.  That is, the first four bytes of the message are in clear and represent the size of the encrypted chunk to follow.  That chunk is then decrypted.  The first four bytes of the decrypted chunk match the size of the rest of the decrypted chunk, which is the message in clear.\r\n\r\nIn terms of code, the handshaking for the server is done in ClientInterface:AuthRunnable.run().  For the client, handshaking is done in ConnectionUtil.getAuthenticatedConnection().\r\n\r\nEncryption of messages, meaning the write side of things, is done in NIOWriteStream.  The instance of SSLEngine that was used to handshake is plumbed into NIOWriteStream.  If SSL is configured, encryption code is in NIOWriteStream's enqueue methods.  If serialization is deferred, then encryption of the message is also deferred.  \r\n\r\nNote that encryption of a single message, or buffer, can result in a series of encrypted buffers that will be written onto the socket.  This concept is captured in a class SSLDeferredSerializationIterator.  Encrypting one message buffer yields an iterator, iterating then yields each encrypted buffer; though there might be just one.\r\n\r\nDecrypting of messages, the read side of things, is done in VoltPort.  VoltPort uses a class, SSLMessageDecrypter, that decrypts a message.  In particular this class is used to read the chunks that make up a message and assemble them into the full message.\r\n\r\nOne final node, the api for DeferredSerialization is changed.  In particular, the serialize method is changed from:\r\n\r\nvoid serialize(ByteBuffer buf) throws IOException;\r\n\r\nto:\r\n\r\nByteBuffer serialize(ByteBuffer buf) throws IOException;\r\n\r\nThe reason for this is that, for messages that will be encrypted, we can't know the serialized size of a message until after it's been encrypted.  In this scheme, when serialize is called, we try to encrypt the message into the given buffer.  If the message doesn't fit, we allocate a new buffer, the serialize() method returns the newly allocated buffer if one was needed, null if not.  The existing code in NIOWriteStreamBase.swapAndSerializeQueuedWrites() has similar logic; if a message will fit in the given buffer it uses that buffer, if not it allocates a new buffer that will be big enough.","created_at":"2016-10-03T21:27:21Z","updated_at":"2016-10-20T21:26:58Z","closed_at":null,"merged_at":null,"merge_commit_sha":"3fb3ccead61fac1becaa1bc98f855dd5f3ed670e","assignee":null,"assignees":[],"milestone":null,"commits_url":"https://api.github.com/repos/VoltDB/voltdb/pulls/3957/commits","review_comments_url":"https://api.github.com/repos/VoltDB/voltdb/pulls/3957/comments","review_comment_url":"https://api.github.com/repos/VoltDB/voltdb/pulls/comments{/number}","comments_url":"https://api.github.com/repos/VoltDB/voltdb/issues/3957/comments","statuses_url":"https://api.github.com/repos/VoltDB/voltdb/statuses/51f9e4a69eee716ce355a4b5b5857e7f8336969a","head":{"label":"VoltDB:ENG-10878","ref":"ENG-10878","sha":"51f9e4a69eee716ce355a4b5b5857e7f8336969a","user":{"login":"VoltDB","id":703910,"avatar_url":"https://avatars.githubusercontent.com/u/703910?v=3","gravatar_id":"","url":"https://api.github.com/users/VoltDB","html_url":"https://github.com/VoltDB","followers_url":"https://api.github.com/users/VoltDB/followers","following_url":"https://api.github.com/users/VoltDB/following{/other_user}","gists_url":"https://api.github.com/users/VoltDB/gists{/gist_id}","starred_url":"https://api.github.com/users/VoltDB/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/VoltDB/subscriptions","organizations_url":"https://api.github.com/users/VoltDB/orgs","repos_url":"https://api.github.com/users/VoltDB/repos","events_url":"https://api.github.com/users/VoltDB/events{/privacy}","received_events_url":"https://api.github.com/users/VoltDB/received_events","type":"Organization","site_admin":false},"repo":{"id":2445970,"name":"voltdb","full_name":"VoltDB/voltdb","owner":{"login":"VoltDB","id":703910,"avatar_url":"https://avatars.githubusercontent.com/u/703910?v=3","gravatar_id":"","url":"https://api.github.com/users/VoltDB","html_url":"https://github.com/VoltDB","followers_url":"https://api.github.com/users/VoltDB/followers","following_url":"https://api.github.com/users/VoltDB/following{/other_user}","gists_url":"https://api.github.com/users/VoltDB/gists{/gist_id}","starred_url":"https://api.github.com/users/VoltDB/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/VoltDB/subscriptions","organizations_url":"https://api.github.com/users/VoltDB/orgs","repos_url":"https://api.github.com/users/VoltDB/repos","events_url":"https://api.github.com/users/VoltDB/events{/privacy}","received_events_url":"https://api.github.com/users/VoltDB/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/VoltDB/voltdb","description":"VoltDB","fork":false,"url":"https://api.github.com/repos/VoltDB/voltdb","forks_url":"https://api.github.com/repos/VoltDB/voltdb/forks","keys_url":"https://api.github.com/repos/VoltDB/voltdb/keys{/key_id}","collaborators_url":"https://api.github.com/repos/VoltDB/voltdb/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/VoltDB/voltdb/teams","hooks_url":"https://api.github.com/repos/VoltDB/voltdb/hooks","issue_events_url":"https://api.github.com/repos/VoltDB/voltdb/issues/events{/number}","events_url":"https://api.github.com/repos/VoltDB/voltdb/events","assignees_url":"https://api.github.com/repos/VoltDB/voltdb/assignees{/user}","branches_url":"https://api.github.com/repos/VoltDB/voltdb/branches{/branch}","tags_url":"https://api.github.com/repos/VoltDB/voltdb/tags","blobs_url":"https://api.github.com/repos/VoltDB/voltdb/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/VoltDB/voltdb/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/VoltDB/voltdb/git/refs{/sha}","trees_url":"https://api.github.com/repos/VoltDB/voltdb/git/trees{/sha}","statuses_url":"https://api.github.com/repos/VoltDB/voltdb/statuses/{sha}","languages_url":"https://api.github.com/repos/VoltDB/voltdb/languages","stargazers_url":"https://api.github.com/repos/VoltDB/voltdb/stargazers","contributors_url":"https://api.github.com/repos/VoltDB/voltdb/contributors","subscribers_url":"https://api.github.com/repos/VoltDB/voltdb/subscribers","subscription_url":"https://api.github.com/repos/VoltDB/voltdb/subscription","commits_url":"https://api.github.com/repos/VoltDB/voltdb/commits{/sha}","git_commits_url":"https://api.github.com/repos/VoltDB/voltdb/git/commits{/sha}","comments_url":"https://api.github.com/repos/VoltDB/voltdb/comments{/number}","issue_comment_url":"https://api.github.com/repos/VoltDB/voltdb/issues/comments{/number}","contents_url":"https://api.github.com/repos/VoltDB/voltdb/contents/{+path}","compare_url":"https://api.github.com/repos/VoltDB/voltdb/compare/{base}...{head}","merges_url":"https://api.github.com/repos/VoltDB/voltdb/merges","archive_url":"https://api.github.com/repos/VoltDB/voltdb/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/VoltDB/voltdb/downloads","issues_url":"https://api.github.com/repos/VoltDB/voltdb/issues{/number}","pulls_url":"https://api.github.com/repos/VoltDB/voltdb/pulls{/number}","milestones_url":"https://api.github.com/repos/VoltDB/voltdb/milestones{/number}","notifications_url":"https://api.github.com/repos/VoltDB/voltdb/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/VoltDB/voltdb/labels{/name}","releases_url":"https://api.github.com/repos/VoltDB/voltdb/releases{/id}","deployments_url":"https://api.github.com/repos/VoltDB/voltdb/deployments","created_at":"2011-09-23T18:46:12Z","updated_at":"2016-10-19T16:04:27Z","pushed_at":"2016-10-20T21:15:10Z","git_url":"git://github.com/VoltDB/voltdb.git","ssh_url":"git@github.com:VoltDB/voltdb.git","clone_url":"https://github.com/VoltDB/voltdb.git","svn_url":"https://github.com/VoltDB/voltdb","homepage":"http://community.voltdb.com/","size":624664,"stargazers_count":879,"watchers_count":879,"language":"Java","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":274,"mirror_url":null,"open_issues_count":62,"forks":274,"open_issues":62,"watchers":879,"default_branch":"master"}},"base":{"label":"VoltDB:master","ref":"master","sha":"4bc05743b43e05c8d59825926ecbc20396b471e5","user":{"login":"VoltDB","id":703910,"avatar_url":"https://avatars.githubusercontent.com/u/703910?v=3","gravatar_id":"","url":"https://api.github.com/users/VoltDB","html_url":"https://github.com/VoltDB","followers_url":"https://api.github.com/users/VoltDB/followers","following_url":"https://api.github.com/users/VoltDB/following{/other_user}","gists_url":"https://api.github.com/users/VoltDB/gists{/gist_id}","starred_url":"https://api.github.com/users/VoltDB/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/VoltDB/subscriptions","organizations_url":"https://api.github.com/users/VoltDB/orgs","repos_url":"https://api.github.com/users/VoltDB/repos","events_url":"https://api.github.com/users/VoltDB/events{/privacy}","received_events_url":"https://api.github.com/users/VoltDB/received_events","type":"Organization","site_admin":false},"repo":{"id":2445970,"name":"voltdb","full_name":"VoltDB/voltdb","owner":{"login":"VoltDB","id":703910,"avatar_url":"https://avatars.githubusercontent.com/u/703910?v=3","gravatar_id":"","url":"https://api.github.com/users/VoltDB","html_url":"https://github.com/VoltDB","followers_url":"https://api.github.com/users/VoltDB/followers","following_url":"https://api.github.com/users/VoltDB/following{/other_user}","gists_url":"https://api.github.com/users/VoltDB/gists{/gist_id}","starred_url":"https://api.github.com/users/VoltDB/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/VoltDB/subscriptions","organizations_url":"https://api.github.com/users/VoltDB/orgs","repos_url":"https://api.github.com/users/VoltDB/repos","events_url":"https://api.github.com/users/VoltDB/events{/privacy}","received_events_url":"https://api.github.com/users/VoltDB/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/VoltDB/voltdb","description":"VoltDB","fork":false,"url":"https://api.github.com/repos/VoltDB/voltdb","forks_url":"https://api.github.com/repos/VoltDB/voltdb/forks","keys_url":"https://api.github.com/repos/VoltDB/voltdb/keys{/key_id}","collaborators_url":"https://api.github.com/repos/VoltDB/voltdb/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/VoltDB/voltdb/teams","hooks_url":"https://api.github.com/repos/VoltDB/voltdb/hooks","issue_events_url":"https://api.github.com/repos/VoltDB/voltdb/issues/events{/number}","events_url":"https://api.github.com/repos/VoltDB/voltdb/events","assignees_url":"https://api.github.com/repos/VoltDB/voltdb/assignees{/user}","branches_url":"https://api.github.com/repos/VoltDB/voltdb/branches{/branch}","tags_url":"https://api.github.com/repos/VoltDB/voltdb/tags","blobs_url":"https://api.github.com/repos/VoltDB/voltdb/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/VoltDB/voltdb/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/VoltDB/voltdb/git/refs{/sha}","trees_url":"https://api.github.com/repos/VoltDB/voltdb/git/trees{/sha}","statuses_url":"https://api.github.com/repos/VoltDB/voltdb/statuses/{sha}","languages_url":"https://api.github.com/repos/VoltDB/voltdb/languages","stargazers_url":"https://api.github.com/repos/VoltDB/voltdb/stargazers","contributors_url":"https://api.github.com/repos/VoltDB/voltdb/contributors","subscribers_url":"https://api.github.com/repos/VoltDB/voltdb/subscribers","subscription_url":"https://api.github.com/repos/VoltDB/voltdb/subscription","commits_url":"https://api.github.com/repos/VoltDB/voltdb/commits{/sha}","git_commits_url":"https://api.github.com/repos/VoltDB/voltdb/git/commits{/sha}","comments_url":"https://api.github.com/repos/VoltDB/voltdb/comments{/number}","issue_comment_url":"https://api.github.com/repos/VoltDB/voltdb/issues/comments{/number}","contents_url":"https://api.github.com/repos/VoltDB/voltdb/contents/{+path}","compare_url":"https://api.github.com/repos/VoltDB/voltdb/compare/{base}...{head}","merges_url":"https://api.github.com/repos/VoltDB/voltdb/merges","archive_url":"https://api.github.com/repos/VoltDB/voltdb/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/VoltDB/voltdb/downloads","issues_url":"https://api.github.com/repos/VoltDB/voltdb/issues{/number}","pulls_url":"https://api.github.com/repos/VoltDB/voltdb/pulls{/number}","milestones_url":"https://api.github.com/repos/VoltDB/voltdb/milestones{/number}","notifications_url":"https://api.github.com/repos/VoltDB/voltdb/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/VoltDB/voltdb/labels{/name}","releases_url":"https://api.github.com/repos/VoltDB/voltdb/releases{/id}","deployments_url":"https://api.github.com/repos/VoltDB/voltdb/deployments","created_at":"2011-09-23T18:46:12Z","updated_at":"2016-10-19T16:04:27Z","pushed_at":"2016-10-20T21:15:10Z","git_url":"git://github.com/VoltDB/voltdb.git","ssh_url":"git@github.com:VoltDB/voltdb.git","clone_url":"https://github.com/VoltDB/voltdb.git","svn_url":"https://github.com/VoltDB/voltdb","homepage":"http://community.voltdb.com/","size":624664,"stargazers_count":879,"watchers_count":879,"language":"Java","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":274,"mirror_url":null,"open_issues_count":62,"forks":274,"open_issues":62,"watchers":879,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/VoltDB/voltdb/pulls/3957"},"html":{"href":"https://github.com/VoltDB/voltdb/pull/3957"},"issue":{"href":"https://api.github.com/repos/VoltDB/voltdb/issues/3957"},"comments":{"href":"https://api.github.com/repos/VoltDB/voltdb/issues/3957/comments"},"review_comments":{"href":"https://api.github.com/repos/VoltDB/voltdb/pulls/3957/comments"},"review_comment":{"href":"https://api.github.com/repos/VoltDB/voltdb/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/VoltDB/voltdb/pulls/3957/commits"},"statuses":{"href":"https://api.github.com/repos/VoltDB/voltdb/statuses/51f9e4a69eee716ce355a4b5b5857e7f8336969a"}}}}
{"actor":{"display_login":"vtkstef"}}