{"action":"created","issue":{"url":"https://api.github.com/repos/crits/crits/issues/660","repository_url":"https://api.github.com/repos/crits/crits","labels_url":"https://api.github.com/repos/crits/crits/issues/660/labels{/name}","comments_url":"https://api.github.com/repos/crits/crits/issues/660/comments","events_url":"https://api.github.com/repos/crits/crits/issues/660/events","html_url":"https://github.com/crits/crits/issues/660","id":118401778,"number":660,"title":"Error uploading file on Microsoft Outlook .MSG File","user":{"login":"myntric","id":6606302,"avatar_url":"https://avatars.githubusercontent.com/u/6606302?v=3","gravatar_id":"","url":"https://api.github.com/users/myntric","html_url":"https://github.com/myntric","followers_url":"https://api.github.com/users/myntric/followers","following_url":"https://api.github.com/users/myntric/following{/other_user}","gists_url":"https://api.github.com/users/myntric/gists{/gist_id}","starred_url":"https://api.github.com/users/myntric/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/myntric/subscriptions","organizations_url":"https://api.github.com/users/myntric/orgs","repos_url":"https://api.github.com/users/myntric/repos","events_url":"https://api.github.com/users/myntric/events{/privacy}","received_events_url":"https://api.github.com/users/myntric/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":10,"created_at":"2015-11-23T15:13:16Z","updated_at":"2016-11-14T15:37:48Z","closed_at":null,"body":"We have had a significant increase in phish messages arriving that contain embedded code that CRITs seems to fail processing each time.  Below is a snippet of the error code logged with Apache2.  This has been an ongoing issue for awhile.  \n\nMon Nov 23 10:12:37.743647 2015] [:error] [pid 8527:tid 140503714576128] 3092311 ERROR: Internal Server Error: /emails/new/outlook/\n[Mon Nov 23 10:12:37.743816 2015] [:error] [pid 8527:tid 140503714576128] Traceback (most recent call last):\n[Mon Nov 23 10:12:37.743888 2015] [:error] [pid 8527:tid 140503714576128]   File \"/usr/lib/python2.7/dist-packages/django/core/handlers/base.py\", line 112, in get_response\n[Mon Nov 23 10:12:37.743898 2015] [:error] [pid 8527:tid 140503714576128]     response = wrapped_callback(request, _callback_args, *_callback_kwargs)\n[Mon Nov 23 10:12:37.743905 2015] [:error] [pid 8527:tid 140503714576128]   File \"/usr/lib/python2.7/dist-packages/django/contrib/auth/decorators.py\", line 22, in _wrapped_view\n[Mon Nov 23 10:12:37.743912 2015] [:error] [pid 8527:tid 140503714576128]     return view_func(request, _args, *_kwargs)\n[Mon Nov 23 10:12:37.743918 2015] [:error] [pid 8527:tid 140503714576128]   File \"/data/crits/crits/emails/views.py\", line 444, in email_outlook_add\n[Mon Nov 23 10:12:37.743925 2015] [:error] [pid 8527:tid 140503714576128]     ticket)\n[Mon Nov 23 10:12:37.743951 2015] [:error] [pid 8527:tid 140503714576128]   File \"/data/crits/crits/emails/handlers.py\", line 783, in handle_msg\n[Mon Nov 23 10:12:37.743959 2015] [:error] [pid 8527:tid 140503714576128]     result = parse_ole_file(data)\n[Mon Nov 23 10:12:37.743966 2015] [:error] [pid 8527:tid 140503714576128]   File \"/data/crits/crits/emails/handlers.py\", line 1518, in parse_ole_file\n[Mon Nov 23 10:12:37.743973 2015] [:error] [pid 8527:tid 140503714576128]     email['raw_body'] = get_stream_data(entry).decode('utf-16')\n[Mon Nov 23 10:12:37.743980 2015] [:error] [pid 8527:tid 140503714576128]   File \"/usr/lib/python2.7/encodings/utf_16.py\", line 16, in decode\n[Mon Nov 23 10:12:37.743988 2015] [:error] [pid 8527:tid 140503714576128]     return codecs.utf_16_decode(input, errors, True)\n[Mon Nov 23 10:12:37.743994 2015] [:error] [pid 8527:tid 140503714576128] UnicodeDecodeError: 'utf16' codec can't decode bytes in position 2-3: illegal encoding\n"},"comment":{"url":"https://api.github.com/repos/crits/crits/issues/comments/260368924","html_url":"https://github.com/crits/crits/issues/660#issuecomment-260368924","issue_url":"https://api.github.com/repos/crits/crits/issues/660","id":260368924,"user":{"login":"myntric","id":6606302,"avatar_url":"https://avatars.githubusercontent.com/u/6606302?v=3","gravatar_id":"","url":"https://api.github.com/users/myntric","html_url":"https://github.com/myntric","followers_url":"https://api.github.com/users/myntric/followers","following_url":"https://api.github.com/users/myntric/following{/other_user}","gists_url":"https://api.github.com/users/myntric/gists{/gist_id}","starred_url":"https://api.github.com/users/myntric/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/myntric/subscriptions","organizations_url":"https://api.github.com/users/myntric/orgs","repos_url":"https://api.github.com/users/myntric/repos","events_url":"https://api.github.com/users/myntric/events{/privacy}","received_events_url":"https://api.github.com/users/myntric/received_events","type":"User","site_admin":false},"created_at":"2016-11-14T15:37:48Z","updated_at":"2016-11-14T15:37:48Z","body":"Adam,\n\nIt took awhile, but I believe I figured out a workaround with the\ndecoding.  I made the following code changes nearly 8 months ago and have\nnot had any issues uploading emails (thousands) since.\n\ndiff /data/backup/crits/emails/handlers.py handlers.py\n1592c1592\n<                 email['subject'] =\nget_stream_data(entry).decode('utf-16',errors='ignore')\n---\n>                 email['subject'] = get_stream_data(entry).decode('utf-16')\n1594c1594\n<                 email['raw_body'] =\nget_stream_data(entry).decode('utf-16',errors='ignore')\n---\n>                 email['raw_body'] =\nget_stream_data(entry).decode('utf-16')\n1596c1596\n<                 email['raw_header'] =\nget_stream_data(entry).decode('utf-16',errors='ignore')\n---\n>                 email['raw_header'] =\nget_stream_data(entry).decode('utf-16')\n1598c1598\n<\nemail['to'].append(get_stream_data(entry).decode('utf-16',errors='ignore').lower())\n---\n>\nemail['to'].append(get_stream_data(entry).decode('utf-16').lower())\n1600c1600\n<                 message_class =\nget_stream_data(entry).decode('utf-16',errors='ignore').lower()\n---\n>                 message_class =\nget_stream_data(entry).decode('utf-16').lower()\n\nI believe there were certain characters contained within the email that\ncouldn't be processed with utf-16, but by ignoring errors in the data\nstream it seems to work.\n\nMike\n\nOn Mon, Nov 30, 2015 at 11:38 AM, Adam Polkosnik [work] <\nnotifications@github.com> wrote:\n\n> Can you try 'utf-16le'?\n>\n> —\n> Reply to this email directly or view it on GitHub\n> <https://github.com/crits/crits/issues/660#issuecomment-160682250>.\n>\n"}}
{"actor":{"display_login":"myntric"}}