{"action":"created","issue":{"url":"https://api.github.com/repos/hexpm/hex/issues/255","repository_url":"https://api.github.com/repos/hexpm/hex","labels_url":"https://api.github.com/repos/hexpm/hex/issues/255/labels{/name}","comments_url":"https://api.github.com/repos/hexpm/hex/issues/255/comments","events_url":"https://api.github.com/repos/hexpm/hex/issues/255/events","html_url":"https://github.com/hexpm/hex/pull/255","id":164215962,"number":255,"title":"Add KeyManager and ContentEncryptor to handle encryption of api key","user":{"login":"potatosalad","id":1804,"avatar_url":"https://avatars.githubusercontent.com/u/1804?v=3","gravatar_id":"","url":"https://api.github.com/users/potatosalad","html_url":"https://github.com/potatosalad","followers_url":"https://api.github.com/users/potatosalad/followers","following_url":"https://api.github.com/users/potatosalad/following{/other_user}","gists_url":"https://api.github.com/users/potatosalad/gists{/gist_id}","starred_url":"https://api.github.com/users/potatosalad/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/potatosalad/subscriptions","organizations_url":"https://api.github.com/users/potatosalad/orgs","repos_url":"https://api.github.com/users/potatosalad/repos","events_url":"https://api.github.com/users/potatosalad/events{/privacy}","received_events_url":"https://api.github.com/users/potatosalad/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":7,"created_at":"2016-07-07T02:23:54Z","updated_at":"2016-07-15T17:53:22Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/hexpm/hex/pulls/255","html_url":"https://github.com/hexpm/hex/pull/255","diff_url":"https://github.com/hexpm/hex/pull/255.diff","patch_url":"https://github.com/hexpm/hex/pull/255.patch"},"body":"I put this together after some questions by @ericmj [yesterday on IRC](https://botbot.me/freenode/elixir-lang/msg/69148630/) about #244.\r\n\r\nHopefully this is helpful and not a headache.\r\n\r\n### Summary of changes\r\n\r\n1. `Hex.Crypto.encrypt/3` and `Hex.Crypto.decrypt/3` are now opaque functions that take a plain/cipher text, password, and tag.  A single binary is returned containing all additional necessary data to authenticate, derive password challenges, and decrypt.\r\n2. Instead of storing both `:key_cipher` and `:key_salt`, a single `:encrypted_key` binary is stored for the encrypted api key.\r\n3. Added `Hex.Crypto.KeyManager` and `Hex.Crypto.ContentEncryptor` to allow for separation between Key Management and Content Encryption algorithms.\r\n4. Altered some of the algorithms to follow guidelines in published standards:\r\n   1. `AES-CBC` replaced with `AES_CBC_HMAC_SHA2` for authenticated content encryption [RFC 7518 Section 5.2.6](https://tools.ietf.org/html/rfc7518#section-5.2.6).\r\n   2. `PBKDF2` replaced with `PBES2_HMAC_SHA2_AES_KW` for key management encryption [RFC 7518 Section 4.8](https://tools.ietf.org/html/rfc7518#section-4.8).\r\n\r\nAdditionally, there is disabled-by-default support for `AES-GCM` lurking in the shadows.  Once support for OTP-17 has been removed, I'd recommend switching to `AES-GCM` for all new encrypted keys.  Older `AES-CBC` encrypted keys can still be decrypted once other new modes are active.\r\n\r\n### How does it work?\r\n\r\n1. A randomly generated `key`, `iv`, and `salt` are used for each encryption operation.\r\n2. A `protected` header is built which contains information about the Key Manager and Content Encryptor used.\r\n3. The `tag` is prepended onto `protected` and used as the `aad` value for the content encryption of the `plain_text` into `cipher_text` and `cipher_tag`.\r\n4. The `key` is encrypted using AES Key Wrap with a key-encryption key derived from PBKDF2 using the `password`, `salt`, and `iterations` into `encrypted_key`.\r\n5. The `protected`, `encrypted_key`, `iv`, `cipher_text`, and `cipher_tag` are encoded as described below.\r\n\r\nThe encoded encryption is a variant of [JWE Compact Serialization](https://tools.ietf.org/html/rfc7516#section-3.1), which is essentially:\r\n\r\n```text\r\nBASE64URL(protected) || '.' ||\r\nBASE64URL(encrypted_key) || '.' ||\r\nBASE64URL(iv) || '.' ||\r\nBASE64URL(cipher_text) || '.' ||\r\nBASE64URL(cipher_tag)\r\n```\r\n\r\nThe `protected` header contains information related to which Key Management and Content Encryption algorithm were used (by default, `PBES2-HS512+A256KW` and `A256CBC-HS512`, respectively).\r\n\r\nFor `PBES2_HMAC_SHA2_AES_KW`, the `protected` header also contains the `salt` and `iterations` values that were used at the time of encryption.\r\n\r\nThe `tag` passed to either `encrypt/3` or `decrypt/3` gets prepended to `protected` and becomes the value used for the Additional Authenticted Data (or `aad`).\r\n\r\nBeyond that, the functions are designed to be opaque.\r\n\r\n##### Example encryption/decryption\r\n\r\n```elixir\r\ncipher_text = Hex.Crypto.encrypt(\"test\", \"password\", \"apikey\")\r\n# => \"VlVWS1JsVjZTWFJUUmsweFRWUkpjbEZVU1RGT2EzUllMbFJFVG1waWJrMTRURmRTTmxWSGJHOWhSVTVMVkRKT2JtVldhekpSVXpWdVVWVkYuVVZSSk1VNXJUa05SZVRGSlZYcFZlRTFuTGc.9sQ46kICihrgzYfnYHziKjJHjqH8vAUlDcXLWfMD9ayzPZafTbqmpYi3DB7Q1A69RnFza1o0LinlKJplCSoWttisoaaKHxG_.2TahiPUFpxVYBcrcLhQKwQ.tj4Wq4qUb9sByt8e7zZCNg.n1Nq8z5vIpabr7JfF3WK597qWOpK7F0ZzZLeQBMdfGw\"\r\n{:ok, plain_text} = Hex.Crypto.decrypt(cipher_text, \"password\", \"apikey\")\r\n# => {:ok, \"test\"}\r\n```\r\n\r\n##### Example encryption without any magic\r\n\r\n```elixir\r\nalias Hex.Crypto.AES_CBC_HMAC_SHA2\r\nalias Hex.Crypto.PBES2_HMAC_SHA2_AES_KW\r\n\r\n# Parameters passed to encrypt/3\r\nplain_text = \"test\"\r\npassword   = \"password\"\r\ntag        = \"apikey\"\r\n\r\n# Numeric Variables\r\nkey_length         = 64\r\niv_length          = 16\r\nderived_key_length = 32\r\nsalt_length        = 16\r\niterations         = Hex.State.fetch!(:pbkdf2_iters) # => 32768\r\n\r\n# Randomly Generated Variables\r\nkey  = :crypto.strong_rand_bytes(key_length)\r\niv   = :crypto.strong_rand_bytes(iv_length)\r\nsalt = :crypto.strong_rand_bytes(salt_length)\r\n\r\n# Build the Protected Header\r\nkey_manager_protected = ([\r\n  \"PBES2-HS512+A256KW\" |> Base.url_encode64(padding: false),\r\n  \".\",\r\n  ([\r\n    salt |> Base.url_encode64(padding: false),\r\n    \".\",\r\n    iterations |> :binary.encode_unsigned(:big) |> Base.url_encode64(padding: false)\r\n  ]\r\n  |> IO.iodata_to_binary()\r\n  |> Base.url_encode64(padding: false))\r\n]\r\n|> IO.iodata_to_binary())\r\n\r\ncontent_encryptor_protected = ([\r\n  \"A256CBC-HS512\" |> Base.url_encode64(padding: false),\r\n  \".\"\r\n]\r\n|> IO.iodata_to_binary())\r\n\r\nprotected = ([\r\n  key_manager_protected |> Base.url_encode64(padding: false),\r\n  \".\",\r\n  content_encryptor_protected |> Base.url_encode64(padding: false)\r\n]\r\n|> IO.iodata_to_binary())\r\n\r\n# Additional Authenticated Data is tag prepended onto protected\r\naad = tag <> protected\r\n\r\n# Content Encryption\r\n{cipher_text, cipher_tag} = AES_CBC_HMAC_SHA2.content_encrypt({aad, plain_text}, key, iv)\r\n\r\n# Key Encryption\r\nencrypted_key = PBES2_HMAC_SHA2_AES_KW.key_encrypt(key, password, salt, iterations, derived_key_length)\r\n\r\n# Encode Encrypted Token\r\nencrypted_token = ([\r\n  protected |> Base.url_encode64(padding: false),\r\n  \".\",\r\n  encrypted_key |> Base.url_encode64(padding: false),\r\n  \".\",\r\n  iv |> Base.url_encode64(padding: false),\r\n  \".\",\r\n  cipher_text |> Base.url_encode64(padding: false),\r\n  \".\",\r\n  cipher_tag |> Base.url_encode64(padding: false)\r\n]\r\n|> IO.iodata_to_binary())\r\n# => \"VlVWS1JsVjZTWFJUUmsweFRWUkpjbEZVU1RGT2EzUllMbFI2VGxoWmEzUlFaVVZPTUZwWFdrbGtia0kyVVcxT1FtSldTVEZSVXpWdVVWVkYuVVZSSk1VNXJUa05SZVRGSlZYcFZlRTFuTGc.AOzy2WFU6_uXRRmbq-jZKNj5trKTYxMQPRssYZ6OVB995ohgeogC5fueefsHtL3yZW-TMrl_IHWtbnwPeumAvS_5tSMoPnAh.LkfGmu9qiIdQ5p2n43Esvg.ZoIJst0LluRwm9VHyesq2w.ZvwBMiFmj_oxYRE0cHkqiNoZ7epeTFHkv5Ln6z5IAkQ\"\r\n```\r\n\r\n##### Example decryption without any magic\r\n\r\n```elixir\r\nalias Hex.Crypto.AES_CBC_HMAC_SHA2\r\nalias Hex.Crypto.PBES2_HMAC_SHA2_AES_KW\r\n\r\n# Parameters passed to decrypt/3\r\ncipher_text = \"VlVWS1JsVjZTWFJUUmsweFRWUkpjbEZVU1RGT2EzUllMbFI2VGxoWmEzUlFaVVZPTUZwWFdrbGtia0kyVVcxT1FtSldTVEZSVXpWdVVWVkYuVVZSSk1VNXJUa05SZVRGSlZYcFZlRTFuTGc.AOzy2WFU6_uXRRmbq-jZKNj5trKTYxMQPRssYZ6OVB995ohgeogC5fueefsHtL3yZW-TMrl_IHWtbnwPeumAvS_5tSMoPnAh.LkfGmu9qiIdQ5p2n43Esvg.ZoIJst0LluRwm9VHyesq2w.ZvwBMiFmj_oxYRE0cHkqiNoZ7epeTFHkv5Ln6z5IAkQ\"\r\npassword    = \"password\"\r\ntag         = \"apikey\"\r\n\r\n# This assumes everything is well-formed and does not do proper error checking.\r\n[\r\n  protected,\r\n  encrypted_key,\r\n  iv,\r\n  cipher_text,\r\n  cipher_tag\r\n] = (cipher_text\r\n|> String.split(\".\")\r\n|> Enum.map(&Base.url_decode64!(&1, padding: false)))\r\n\r\n[\r\n  key_manager_protected,\r\n  content_encryptor_protected\r\n] = (protected\r\n|> String.split(\".\")\r\n|> Enum.map(&Base.url_decode64!(&1, padding: false)))\r\n\r\n# Decode the Key Manager and its parameters\r\n[\r\n  \"PBES2-HS512+A256KW\",\r\n  key_manager_params\r\n] = (key_manager_protected\r\n|> String.split(\".\")\r\n|> Enum.map(&Base.url_decode64!(&1, padding: false)))\r\n\r\nderived_key_length = 32 # based on PBES2-HS512+A256KW matched above\r\n\r\n[\r\n  salt,\r\n  iterations\r\n] = (key_manager_params\r\n|> String.split(\".\")\r\n|> Enum.map(&Base.url_decode64!(&1, padding: false)))\r\n\r\niterations = :binary.decode_unsigned(iterations, :big)\r\n\r\n# Decode the Content Encryptor and its parameters\r\n[\r\n  \"A256CBC-HS512\",\r\n  \"\"\r\n] = (content_encryptor_protected\r\n|> String.split(\".\")\r\n|> Enum.map(&Base.url_decode64!(&1, padding: false)))\r\n\r\n# Additional Authenticated Data is tag prepended onto protected\r\naad = tag <> protected\r\n\r\n# Key Decryption\r\n{:ok, key} = PBES2_HMAC_SHA2_AES_KW.key_decrypt(encrypted_key, password, salt, iterations, derived_key_length)\r\n\r\n# Content Decryption\r\n{:ok, plain_text} = AES_CBC_HMAC_SHA2.content_decrypt({aad, cipher_text, cipher_tag}, key, iv)\r\n# => {:ok, \"test\"}\r\n```\r\n\r\n### Future brainstorming\r\n\r\nBy separating the content encryption and key management operations, these design changes should allow for future algorithms to easily be added while maintaining backwards compatibility with older algorithms.\r\n\r\nThis would also allow, for example, support for encryption based on SSH keys (RSA, EC, and Ed25519) rather than a password alone (although the keys themselves can be encrypted externally with a password)."},"comment":{"url":"https://api.github.com/repos/hexpm/hex/issues/comments/233021486","html_url":"https://github.com/hexpm/hex/pull/255#issuecomment-233021486","issue_url":"https://api.github.com/repos/hexpm/hex/issues/255","id":233021486,"user":{"login":"ericmj","id":316890,"avatar_url":"https://avatars.githubusercontent.com/u/316890?v=3","gravatar_id":"","url":"https://api.github.com/users/ericmj","html_url":"https://github.com/ericmj","followers_url":"https://api.github.com/users/ericmj/followers","following_url":"https://api.github.com/users/ericmj/following{/other_user}","gists_url":"https://api.github.com/users/ericmj/gists{/gist_id}","starred_url":"https://api.github.com/users/ericmj/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ericmj/subscriptions","organizations_url":"https://api.github.com/users/ericmj/orgs","repos_url":"https://api.github.com/users/ericmj/repos","events_url":"https://api.github.com/users/ericmj/events{/privacy}","received_events_url":"https://api.github.com/users/ericmj/received_events","type":"User","site_admin":false},"created_at":"2016-07-15T17:53:22Z","updated_at":"2016-07-15T17:53:22Z","body":"> I forgot to mention: if you'd prefer the Direct Encryption route, I can modify the code so it does that instead.\r\n\r\nLet's do that and use `:erlang.term_to_binary`. Should reduce a lot of complexity. Thank you!"}}
{"actor":{"display_login":"ericmj"}}