{"action":"created","comment":{"url":"https://api.github.com/repos/allenai/allennlp/pulls/comments/221068037","pull_request_review_id":159629773,"id":221068037,"node_id":"MDI0OlB1bGxSZXF1ZXN0UmV2aWV3Q29tbWVudDIyMTA2ODAzNw==","diff_hunk":"@@ -0,0 +1,214 @@\n+from typing import Sequence, Dict, List\n+\n+import torch\n+import numpy as np\n+\n+from allennlp.common.checks import ConfigurationError\n+from allennlp.modules.highway import Highway\n+from allennlp.modules.masked_layer_norm import MaskedLayerNorm\n+from allennlp.modules.token_embedders import TokenEmbedder\n+from allennlp.nn.util import add_sentence_boundary_token_ids\n+\n+_VALID_PROJECTION_LOCATIONS = {'after_cnn', 'after_highway', None}\n+\n+@TokenEmbedder.register('cnn-highway')\n+class CnnHighwayEncoder(TokenEmbedder):\n+    \"\"\"\n+    The character CNN + highway encoder from Kim et al \"Character aware neural language models\"\n+    https://arxiv.org/abs/1508.06615\n+    with an optional projection.\n+\n+    Parameters\n+    ----------\n+    embedding_dim: int\n+        The dimension of the initial character embedding.\n+    filters: ``Sequence[Sequence[int]]``\n+        A sequence of pairs (filter_width, num_filters).\n+    num_highway: int\n+        The number of highway layers.\n+    projection_dim: int\n+        The output dimension of the projection layer.\n+    activation: str, optional (default = 'relu')\n+        The activation function for the convolutional layers.\n+    max_characters_per_token: int, optional (default = 50)\n+        The maximum length of any token.\n+    num_characters: int, optional (default = 262)\n+        The number of characters in the vocabulary.\n+    projection_location: str, optional (default = 'after_highway')\n+        Where to apply the projection layer. Valid values are\n+        'after_highway', 'after_cnn', and None.\n+    do_layer_norm: bool, optional (default = False)\n+        If True, we apply ``MaskedLayerNorm`` to the final encoded result.\n+    bos_characters: ``List[int]``, optional (default = None)","path":"allennlp/modules/token_embedders/cnn_highway_encoder.py","position":42,"original_position":42,"commit_id":"fd7f583d750f59b62cdf919a64d6ad3656ad14a1","original_commit_id":"fd7f583d750f59b62cdf919a64d6ad3656ad14a1","user":{"login":"matt-gardner","id":3291951,"node_id":"MDQ6VXNlcjMyOTE5NTE=","avatar_url":"https://avatars2.githubusercontent.com/u/3291951?v=4","gravatar_id":"","url":"https://api.github.com/users/matt-gardner","html_url":"https://github.com/matt-gardner","followers_url":"https://api.github.com/users/matt-gardner/followers","following_url":"https://api.github.com/users/matt-gardner/following{/other_user}","gists_url":"https://api.github.com/users/matt-gardner/gists{/gist_id}","starred_url":"https://api.github.com/users/matt-gardner/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/matt-gardner/subscriptions","organizations_url":"https://api.github.com/users/matt-gardner/orgs","repos_url":"https://api.github.com/users/matt-gardner/repos","events_url":"https://api.github.com/users/matt-gardner/events{/privacy}","received_events_url":"https://api.github.com/users/matt-gardner/received_events","type":"User","site_admin":false},"body":"Maybe easier to talk about this in person?","created_at":"2018-09-27T20:34:39Z","updated_at":"2018-09-27T20:34:39Z","html_url":"https://github.com/allenai/allennlp/pull/1787#discussion_r221068037","pull_request_url":"https://api.github.com/repos/allenai/allennlp/pulls/1787","author_association":"MEMBER","_links":{"self":{"href":"https://api.github.com/repos/allenai/allennlp/pulls/comments/221068037"},"html":{"href":"https://github.com/allenai/allennlp/pull/1787#discussion_r221068037"},"pull_request":{"href":"https://api.github.com/repos/allenai/allennlp/pulls/1787"}},"in_reply_to_id":220419830},"pull_request":{"url":"https://api.github.com/repos/allenai/allennlp/pulls/1787","id":216480459,"node_id":"MDExOlB1bGxSZXF1ZXN0MjE2NDgwNDU5","html_url":"https://github.com/allenai/allennlp/pull/1787","diff_url":"https://github.com/allenai/allennlp/pull/1787.diff","patch_url":"https://github.com/allenai/allennlp/pull/1787.patch","issue_url":"https://api.github.com/repos/allenai/allennlp/issues/1787","number":1787,"state":"open","locked":false,"title":"contextual encoder + character encoder (from calypso)","user":{"login":"joelgrus","id":1308313,"node_id":"MDQ6VXNlcjEzMDgzMTM=","avatar_url":"https://avatars1.githubusercontent.com/u/1308313?v=4","gravatar_id":"","url":"https://api.github.com/users/joelgrus","html_url":"https://github.com/joelgrus","followers_url":"https://api.github.com/users/joelgrus/followers","following_url":"https://api.github.com/users/joelgrus/following{/other_user}","gists_url":"https://api.github.com/users/joelgrus/gists{/gist_id}","starred_url":"https://api.github.com/users/joelgrus/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/joelgrus/subscriptions","organizations_url":"https://api.github.com/users/joelgrus/orgs","repos_url":"https://api.github.com/users/joelgrus/repos","events_url":"https://api.github.com/users/joelgrus/events{/privacy}","received_events_url":"https://api.github.com/users/joelgrus/received_events","type":"User","site_admin":false},"body":"ok, based on early feedback, I made two big changes\r\n\r\n(1) I got rid of the `ContextualEncoder` abstraction, since as mattg pointed out it was basically just `Seq2SeqEncoder`. accordingly, I made the specific `ContextualEncoder` subclasses (which might only be one at this point) `Seq2SeqEncoder` subclasses instead\r\n(1a) after doing that I was able to rename `ContextualEncoderWrapper` to `ContextualEncoder`\r\n(1b) I got rid of the SruEncoder, as it's still experimental\r\n\r\n(2) as mattp pointed out, the calypso `CharacterEncoder` was very very similar to the existing `_ElmoCharacterEncoder`, so eventually I was able to rip out the `_ElmoCharacterEncoder` and just use the calypso one. that required some amount of weird plumbing, but I think it's all correct (all the tests pass, at least)\r\n\r\nI haven't touched all the transformer pieces of the calypso contextual embeddings, I think I should put that in a different PR. also it's possible that the locations of things here are not optimal, I'm open to feedback on that point.","created_at":"2018-09-19T00:20:55Z","updated_at":"2018-09-27T20:34:39Z","closed_at":null,"merged_at":null,"merge_commit_sha":"c18aecd05fc2ce5eaa2cc560a085c9ecc3503956","assignee":null,"assignees":[],"requested_reviewers":[],"requested_teams":[],"labels":[],"milestone":null,"commits_url":"https://api.github.com/repos/allenai/allennlp/pulls/1787/commits","review_comments_url":"https://api.github.com/repos/allenai/allennlp/pulls/1787/comments","review_comment_url":"https://api.github.com/repos/allenai/allennlp/pulls/comments{/number}","comments_url":"https://api.github.com/repos/allenai/allennlp/issues/1787/comments","statuses_url":"https://api.github.com/repos/allenai/allennlp/statuses/fd7f583d750f59b62cdf919a64d6ad3656ad14a1","head":{"label":"joelgrus:cew","ref":"cew","sha":"fd7f583d750f59b62cdf919a64d6ad3656ad14a1","user":{"login":"joelgrus","id":1308313,"node_id":"MDQ6VXNlcjEzMDgzMTM=","avatar_url":"https://avatars1.githubusercontent.com/u/1308313?v=4","gravatar_id":"","url":"https://api.github.com/users/joelgrus","html_url":"https://github.com/joelgrus","followers_url":"https://api.github.com/users/joelgrus/followers","following_url":"https://api.github.com/users/joelgrus/following{/other_user}","gists_url":"https://api.github.com/users/joelgrus/gists{/gist_id}","starred_url":"https://api.github.com/users/joelgrus/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/joelgrus/subscriptions","organizations_url":"https://api.github.com/users/joelgrus/orgs","repos_url":"https://api.github.com/users/joelgrus/repos","events_url":"https://api.github.com/users/joelgrus/events{/privacy}","received_events_url":"https://api.github.com/users/joelgrus/received_events","type":"User","site_admin":false},"repo":{"id":114700287,"node_id":"MDEwOlJlcG9zaXRvcnkxMTQ3MDAyODc=","name":"allennlp","full_name":"joelgrus/allennlp","private":false,"owner":{"login":"joelgrus","id":1308313,"node_id":"MDQ6VXNlcjEzMDgzMTM=","avatar_url":"https://avatars1.githubusercontent.com/u/1308313?v=4","gravatar_id":"","url":"https://api.github.com/users/joelgrus","html_url":"https://github.com/joelgrus","followers_url":"https://api.github.com/users/joelgrus/followers","following_url":"https://api.github.com/users/joelgrus/following{/other_user}","gists_url":"https://api.github.com/users/joelgrus/gists{/gist_id}","starred_url":"https://api.github.com/users/joelgrus/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/joelgrus/subscriptions","organizations_url":"https://api.github.com/users/joelgrus/orgs","repos_url":"https://api.github.com/users/joelgrus/repos","events_url":"https://api.github.com/users/joelgrus/events{/privacy}","received_events_url":"https://api.github.com/users/joelgrus/received_events","type":"User","site_admin":false},"html_url":"https://github.com/joelgrus/allennlp","description":"An open-source NLP research library, built on PyTorch.","fork":true,"url":"https://api.github.com/repos/joelgrus/allennlp","forks_url":"https://api.github.com/repos/joelgrus/allennlp/forks","keys_url":"https://api.github.com/repos/joelgrus/allennlp/keys{/key_id}","collaborators_url":"https://api.github.com/repos/joelgrus/allennlp/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/joelgrus/allennlp/teams","hooks_url":"https://api.github.com/repos/joelgrus/allennlp/hooks","issue_events_url":"https://api.github.com/repos/joelgrus/allennlp/issues/events{/number}","events_url":"https://api.github.com/repos/joelgrus/allennlp/events","assignees_url":"https://api.github.com/repos/joelgrus/allennlp/assignees{/user}","branches_url":"https://api.github.com/repos/joelgrus/allennlp/branches{/branch}","tags_url":"https://api.github.com/repos/joelgrus/allennlp/tags","blobs_url":"https://api.github.com/repos/joelgrus/allennlp/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/joelgrus/allennlp/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/joelgrus/allennlp/git/refs{/sha}","trees_url":"https://api.github.com/repos/joelgrus/allennlp/git/trees{/sha}","statuses_url":"https://api.github.com/repos/joelgrus/allennlp/statuses/{sha}","languages_url":"https://api.github.com/repos/joelgrus/allennlp/languages","stargazers_url":"https://api.github.com/repos/joelgrus/allennlp/stargazers","contributors_url":"https://api.github.com/repos/joelgrus/allennlp/contributors","subscribers_url":"https://api.github.com/repos/joelgrus/allennlp/subscribers","subscription_url":"https://api.github.com/repos/joelgrus/allennlp/subscription","commits_url":"https://api.github.com/repos/joelgrus/allennlp/commits{/sha}","git_commits_url":"https://api.github.com/repos/joelgrus/allennlp/git/commits{/sha}","comments_url":"https://api.github.com/repos/joelgrus/allennlp/comments{/number}","issue_comment_url":"https://api.github.com/repos/joelgrus/allennlp/issues/comments{/number}","contents_url":"https://api.github.com/repos/joelgrus/allennlp/contents/{+path}","compare_url":"https://api.github.com/repos/joelgrus/allennlp/compare/{base}...{head}","merges_url":"https://api.github.com/repos/joelgrus/allennlp/merges","archive_url":"https://api.github.com/repos/joelgrus/allennlp/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/joelgrus/allennlp/downloads","issues_url":"https://api.github.com/repos/joelgrus/allennlp/issues{/number}","pulls_url":"https://api.github.com/repos/joelgrus/allennlp/pulls{/number}","milestones_url":"https://api.github.com/repos/joelgrus/allennlp/milestones{/number}","notifications_url":"https://api.github.com/repos/joelgrus/allennlp/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/joelgrus/allennlp/labels{/name}","releases_url":"https://api.github.com/repos/joelgrus/allennlp/releases{/id}","deployments_url":"https://api.github.com/repos/joelgrus/allennlp/deployments","created_at":"2017-12-19T00:12:40Z","updated_at":"2017-12-19T00:12:43Z","pushed_at":"2018-09-25T23:22:16Z","git_url":"git://github.com/joelgrus/allennlp.git","ssh_url":"git@github.com:joelgrus/allennlp.git","clone_url":"https://github.com/joelgrus/allennlp.git","svn_url":"https://github.com/joelgrus/allennlp","homepage":"http://www.allennlp.org/","size":52241,"stargazers_count":0,"watchers_count":0,"language":"Python","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":0,"mirror_url":null,"archived":false,"open_issues_count":0,"license":{"key":"apache-2.0","name":"Apache License 2.0","spdx_id":"Apache-2.0","url":"https://api.github.com/licenses/apache-2.0","node_id":"MDc6TGljZW5zZTI="},"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"allenai:master","ref":"master","sha":"8be358e9a495b0fad6202cca89ce29a5f07dc504","user":{"login":"allenai","id":5667695,"node_id":"MDEyOk9yZ2FuaXphdGlvbjU2Njc2OTU=","avatar_url":"https://avatars3.githubusercontent.com/u/5667695?v=4","gravatar_id":"","url":"https://api.github.com/users/allenai","html_url":"https://github.com/allenai","followers_url":"https://api.github.com/users/allenai/followers","following_url":"https://api.github.com/users/allenai/following{/other_user}","gists_url":"https://api.github.com/users/allenai/gists{/gist_id}","starred_url":"https://api.github.com/users/allenai/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/allenai/subscriptions","organizations_url":"https://api.github.com/users/allenai/orgs","repos_url":"https://api.github.com/users/allenai/repos","events_url":"https://api.github.com/users/allenai/events{/privacy}","received_events_url":"https://api.github.com/users/allenai/received_events","type":"Organization","site_admin":false},"repo":{"id":91356408,"node_id":"MDEwOlJlcG9zaXRvcnk5MTM1NjQwOA==","name":"allennlp","full_name":"allenai/allennlp","private":false,"owner":{"login":"allenai","id":5667695,"node_id":"MDEyOk9yZ2FuaXphdGlvbjU2Njc2OTU=","avatar_url":"https://avatars3.githubusercontent.com/u/5667695?v=4","gravatar_id":"","url":"https://api.github.com/users/allenai","html_url":"https://github.com/allenai","followers_url":"https://api.github.com/users/allenai/followers","following_url":"https://api.github.com/users/allenai/following{/other_user}","gists_url":"https://api.github.com/users/allenai/gists{/gist_id}","starred_url":"https://api.github.com/users/allenai/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/allenai/subscriptions","organizations_url":"https://api.github.com/users/allenai/orgs","repos_url":"https://api.github.com/users/allenai/repos","events_url":"https://api.github.com/users/allenai/events{/privacy}","received_events_url":"https://api.github.com/users/allenai/received_events","type":"Organization","site_admin":false},"html_url":"https://github.com/allenai/allennlp","description":"An open-source NLP research library, built on PyTorch.","fork":false,"url":"https://api.github.com/repos/allenai/allennlp","forks_url":"https://api.github.com/repos/allenai/allennlp/forks","keys_url":"https://api.github.com/repos/allenai/allennlp/keys{/key_id}","collaborators_url":"https://api.github.com/repos/allenai/allennlp/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/allenai/allennlp/teams","hooks_url":"https://api.github.com/repos/allenai/allennlp/hooks","issue_events_url":"https://api.github.com/repos/allenai/allennlp/issues/events{/number}","events_url":"https://api.github.com/repos/allenai/allennlp/events","assignees_url":"https://api.github.com/repos/allenai/allennlp/assignees{/user}","branches_url":"https://api.github.com/repos/allenai/allennlp/branches{/branch}","tags_url":"https://api.github.com/repos/allenai/allennlp/tags","blobs_url":"https://api.github.com/repos/allenai/allennlp/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/allenai/allennlp/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/allenai/allennlp/git/refs{/sha}","trees_url":"https://api.github.com/repos/allenai/allennlp/git/trees{/sha}","statuses_url":"https://api.github.com/repos/allenai/allennlp/statuses/{sha}","languages_url":"https://api.github.com/repos/allenai/allennlp/languages","stargazers_url":"https://api.github.com/repos/allenai/allennlp/stargazers","contributors_url":"https://api.github.com/repos/allenai/allennlp/contributors","subscribers_url":"https://api.github.com/repos/allenai/allennlp/subscribers","subscription_url":"https://api.github.com/repos/allenai/allennlp/subscription","commits_url":"https://api.github.com/repos/allenai/allennlp/commits{/sha}","git_commits_url":"https://api.github.com/repos/allenai/allennlp/git/commits{/sha}","comments_url":"https://api.github.com/repos/allenai/allennlp/comments{/number}","issue_comment_url":"https://api.github.com/repos/allenai/allennlp/issues/comments{/number}","contents_url":"https://api.github.com/repos/allenai/allennlp/contents/{+path}","compare_url":"https://api.github.com/repos/allenai/allennlp/compare/{base}...{head}","merges_url":"https://api.github.com/repos/allenai/allennlp/merges","archive_url":"https://api.github.com/repos/allenai/allennlp/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/allenai/allennlp/downloads","issues_url":"https://api.github.com/repos/allenai/allennlp/issues{/number}","pulls_url":"https://api.github.com/repos/allenai/allennlp/pulls{/number}","milestones_url":"https://api.github.com/repos/allenai/allennlp/milestones{/number}","notifications_url":"https://api.github.com/repos/allenai/allennlp/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/allenai/allennlp/labels{/name}","releases_url":"https://api.github.com/repos/allenai/allennlp/releases{/id}","deployments_url":"https://api.github.com/repos/allenai/allennlp/deployments","created_at":"2017-05-15T15:52:41Z","updated_at":"2018-09-27T19:44:08Z","pushed_at":"2018-09-27T20:24:07Z","git_url":"git://github.com/allenai/allennlp.git","ssh_url":"git@github.com:allenai/allennlp.git","clone_url":"https://github.com/allenai/allennlp.git","svn_url":"https://github.com/allenai/allennlp","homepage":"http://www.allennlp.org/","size":51429,"stargazers_count":3426,"watchers_count":3426,"language":"Python","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":606,"mirror_url":null,"archived":false,"open_issues_count":97,"license":{"key":"apache-2.0","name":"Apache License 2.0","spdx_id":"Apache-2.0","url":"https://api.github.com/licenses/apache-2.0","node_id":"MDc6TGljZW5zZTI="},"forks":606,"open_issues":97,"watchers":3426,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/allenai/allennlp/pulls/1787"},"html":{"href":"https://github.com/allenai/allennlp/pull/1787"},"issue":{"href":"https://api.github.com/repos/allenai/allennlp/issues/1787"},"comments":{"href":"https://api.github.com/repos/allenai/allennlp/issues/1787/comments"},"review_comments":{"href":"https://api.github.com/repos/allenai/allennlp/pulls/1787/comments"},"review_comment":{"href":"https://api.github.com/repos/allenai/allennlp/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/allenai/allennlp/pulls/1787/commits"},"statuses":{"href":"https://api.github.com/repos/allenai/allennlp/statuses/fd7f583d750f59b62cdf919a64d6ad3656ad14a1"}},"author_association":"CONTRIBUTOR"}}
{"actor":{"display_login":"matt-gardner"}}