{"action":"created","issue":{"url":"https://api.github.com/repos/google/timesketch/issues/156","repository_url":"https://api.github.com/repos/google/timesketch","labels_url":"https://api.github.com/repos/google/timesketch/issues/156/labels{/name}","comments_url":"https://api.github.com/repos/google/timesketch/issues/156/comments","events_url":"https://api.github.com/repos/google/timesketch/issues/156/events","html_url":"https://github.com/google/timesketch/issues/156","id":121025256,"number":156,"title":"Plaso Upload Issues","user":{"login":"landw1re","id":7854165,"avatar_url":"https://avatars.githubusercontent.com/u/7854165?v=3","gravatar_id":"","url":"https://api.github.com/users/landw1re","html_url":"https://github.com/landw1re","followers_url":"https://api.github.com/users/landw1re/followers","following_url":"https://api.github.com/users/landw1re/following{/other_user}","gists_url":"https://api.github.com/users/landw1re/gists{/gist_id}","starred_url":"https://api.github.com/users/landw1re/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/landw1re/subscriptions","organizations_url":"https://api.github.com/users/landw1re/orgs","repos_url":"https://api.github.com/users/landw1re/repos","events_url":"https://api.github.com/users/landw1re/events{/privacy}","received_events_url":"https://api.github.com/users/landw1re/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/google/timesketch/labels/bug","name":"bug","color":"fc2929"},{"url":"https://api.github.com/repos/google/timesketch/labels/wontfix","name":"wontfix","color":"ffffff"}],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":7,"created_at":"2015-12-08T15:01:48Z","updated_at":"2016-04-26T11:25:43Z","closed_at":"2016-04-08T08:04:13Z","body":"I've just installed all the dependencies and Timesketch on a Ubuntu 14.04LTS box. Everything built OK without issue and Redis, Elasticsearch, Celery & Timesketch start-up fine. The issue that I'm having is with the Uploading of Plaso files. I tried it with a small (639K) .plaso file to test things out. I created the small file from a larger (4GB) plaso file by using the following psort.py command:\r\n\r\n```python\r\npsort.py -q -o pstorage -w testing.plaso --slice \"2015-01-08 09:47:16\" --slice_size 100 timeline.plaso\r\n```\r\n**Note**: the psort.py command runs clean and I don't receive any errors. The file doesn't appear to be corrupted. I changed the output to l2tcsv just to make sure there was data and it produced a 6.0M csv with content.\r\n\r\nWhen I upload the small file I keep getting the following errors:\r\n### Error from tsctl runserver\r\n```python\r\n[2015-12-08 14:37:09,173: INFO/MainProcess] Received task: timesketch.lib.tasks.run_plaso[e604c57e4f8440ac8c05f103a606f242]\r\n127.0.0.1 - - [08/Dec/2015 14:37:09] \"POST /api/v1/upload/ HTTP/1.1\" 201 -\r\n[2015-12-08 14:37:09,223: INFO/Worker-8] Index: e604c57e4f8440ac8c05f103a606f242\r\n[2015-12-08 14:37:09,224: INFO/Worker-8] Timeline name: testing\r\n[2015-12-08 14:37:09,236: INFO/Worker-8] Starting new HTTP connection (1): 127.0.0.1\r\n[2015-12-08 14:37:09,273: ERROR/MainProcess] Task timesketch.lib.tasks.run_plaso[e604c57e4f8440ac8c05f103a606f242] raised unexpected: RequestError(400, u'mapper_parsing_exception', {u'status': 400, u'error': {u'type': u'mapper_parsing_exception', u'root_cause': [{u'reason': u'Mapping definition for [_timestamp] has unsupported parameters:  [path : datetime]', u'type': u'mapper_parsing_exception'}], u'caused_by': {u'reason': u'Mapping definition for [_timestamp] has unsupported parameters:  [path : datetime]', u'type': u'mapper_parsing_exception'}, u'reason': u'Failed to parse mapping [plaso_event]: Mapping definition for [_timestamp] has unsupported parameters:  [path : datetime]'}})\r\nTraceback (most recent call last):\r\n  File \"/usr/local/lib/python2.7/dist-packages/celery/app/trace.py\", line 240, in trace_task\r\n    R = retval = fun(*args, **kwargs)\r\n  File \"/usr/local/lib/python2.7/dist-packages/timesketch/__init__.py\", line 162, in __call__\r\n    return TaskBase.__call__(self, *args, **kwargs)\r\n  File \"/usr/local/lib/python2.7/dist-packages/celery/app/trace.py\", line 438, in __protected_call__\r\n    return self.run(*args, **kwargs)\r\n  File \"/usr/local/lib/python2.7/dist-packages/timesketch/lib/tasks.py\", line 79, in run_plaso\r\n    output_module, storage_file, plugins, queue_producers)\r\n  File \"/usr/lib/python2.7/dist-packages/plaso/frontend/psort.py\", line 417, in ProcessStorage\r\n    output_module, deduplicate_events)\r\n  File \"/usr/lib/python2.7/dist-packages/plaso/output/interface.py\", line 167, in __init__\r\n    self._output_module.WriteHeader()\r\n  File \"/usr/lib/python2.7/dist-packages/plaso/output/timesketch_out.py\", line 213, in WriteHeader\r\n    index=self._index_name, body={u'mappings': _document_mapping})\r\n  File \"/usr/local/lib/python2.7/dist-packages/elasticsearch/client/utils.py\", line 69, in _wrapped\r\n    return func(*args, params=params, **kwargs)\r\n  File \"/usr/local/lib/python2.7/dist-packages/elasticsearch/client/indices.py\", line 105, in create\r\n    params=params, body=body)\r\n  File \"/usr/local/lib/python2.7/dist-packages/elasticsearch/transport.py\", line 307, in perform_request\r\n    status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)\r\n  File \"/usr/local/lib/python2.7/dist-packages/elasticsearch/connection/http_urllib3.py\", line 93, in perform_request\r\n    self._raise_error(response.status, raw_data)\r\n  File \"/usr/local/lib/python2.7/dist-packages/elasticsearch/connection/base.py\", line 105, in _raise_error\r\n    raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)\r\nRequestError: TransportError(400, u'mapper_parsing_exception')\r\n```\r\n### Error produced by Elasticsearch\r\n```java\r\n[2015-12-08 14:37:09,267][DEBUG][action.admin.indices.create] [Guido Carosella] [e604c57e4f8440ac8c05f103a606f242] failed to create\r\nMapperParsingException[Failed to parse mapping [plaso_event]: Mapping definition for [_timestamp] has unsupported parameters:  [path : datetime]]; nested: MapperParsingException[Mapping definition for [_timestamp] has unsupported parameters:  [path : datetime]];\r\n        at org.elasticsearch.cluster.metadata.MetaDataCreateIndexService$2.execute(MetaDataCreateIndexService.java:381)\r\n        at org.elasticsearch.cluster.service.InternalClusterService$UpdateTask.run(InternalClusterService.java:388)\r\n        at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:231)\r\n        at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:194)\r\n        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)\r\n        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)\r\n        at java.lang.Thread.run(Thread.java:745)\r\nCaused by: MapperParsingException[Mapping definition for [_timestamp] has unsupported parameters:  [path : datetime]]\r\n        at org.elasticsearch.index.mapper.DocumentMapperParser.checkNoRemainingFields(DocumentMapperParser.java:266)\r\n        at org.elasticsearch.index.mapper.DocumentMapperParser.checkNoRemainingFields(DocumentMapperParser.java:260)\r\n        at org.elasticsearch.index.mapper.DocumentMapperParser.parse(DocumentMapperParser.java:243)\r\n        at org.elasticsearch.index.mapper.DocumentMapperParser.parseCompressed(DocumentMapperParser.java:191)\r\n        at org.elasticsearch.index.mapper.MapperService.parse(MapperService.java:387)\r\n        at org.elasticsearch.index.mapper.MapperService.merge(MapperService.java:261)\r\n        at org.elasticsearch.cluster.metadata.MetaDataCreateIndexService$2.execute(MetaDataCreateIndexService.java:378)\r\n        ... 6 more\r\n[2015-12-08 14:37:09,268][INFO ][rest.suppressed          ] /e604c57e4f8440ac8c05f103a606f242 Params: {index=e604c57e4f8440ac8c05f103a606f242}\r\nMapperParsingException[Failed to parse mapping [plaso_event]: Mapping definition for [_timestamp] has unsupported parameters:  [path : datetime]]; nested: MapperParsingException[Mapping definition for [_timestamp] has unsupported parameters:  [path : datetime]];\r\n        at org.elasticsearch.cluster.metadata.MetaDataCreateIndexService$2.execute(MetaDataCreateIndexService.java:381)\r\n        at org.elasticsearch.cluster.service.InternalClusterService$UpdateTask.run(InternalClusterService.java:388)\r\n        at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:231)\r\n        at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:194)\r\n        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)\r\n        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)\r\n        at java.lang.Thread.run(Thread.java:745)\r\nCaused by: MapperParsingException[Mapping definition for [_timestamp] has unsupported parameters:  [path : datetime]]\r\n        at org.elasticsearch.index.mapper.DocumentMapperParser.checkNoRemainingFields(DocumentMapperParser.java:266)\r\n        at org.elasticsearch.index.mapper.DocumentMapperParser.checkNoRemainingFields(DocumentMapperParser.java:260)\r\n        at org.elasticsearch.index.mapper.DocumentMapperParser.parse(DocumentMapperParser.java:243)\r\n        at org.elasticsearch.index.mapper.DocumentMapperParser.parseCompressed(DocumentMapperParser.java:191)\r\n        at org.elasticsearch.index.mapper.MapperService.parse(MapperService.java:387)\r\n        at org.elasticsearch.index.mapper.MapperService.merge(MapperService.java:261)\r\n        at org.elasticsearch.cluster.metadata.MetaDataCreateIndexService$2.execute(MetaDataCreateIndexService.java:378)\r\n        ... 6 more\r\n```"},"comment":{"url":"https://api.github.com/repos/google/timesketch/issues/comments/214709820","html_url":"https://github.com/google/timesketch/issues/156#issuecomment-214709820","issue_url":"https://api.github.com/repos/google/timesketch/issues/156","id":214709820,"user":{"login":"berggren","id":316362,"avatar_url":"https://avatars.githubusercontent.com/u/316362?v=3","gravatar_id":"","url":"https://api.github.com/users/berggren","html_url":"https://github.com/berggren","followers_url":"https://api.github.com/users/berggren/followers","following_url":"https://api.github.com/users/berggren/following{/other_user}","gists_url":"https://api.github.com/users/berggren/gists{/gist_id}","starred_url":"https://api.github.com/users/berggren/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/berggren/subscriptions","organizations_url":"https://api.github.com/users/berggren/orgs","repos_url":"https://api.github.com/users/berggren/repos","events_url":"https://api.github.com/users/berggren/events{/privacy}","received_events_url":"https://api.github.com/users/berggren/received_events","type":"User","site_admin":false},"created_at":"2016-04-26T11:25:43Z","updated_at":"2016-04-26T11:25:43Z","body":"Hi,\n\nYes the fix is not yet pushed yet (any time now :). It will be part of the\n1.5 Plaso release.\nhttps://github.com/log2timeline/plaso/issues/607\n\nKind regards,\nJohan\n\nOn Sat, Apr 23, 2016 at 5:48 PM enhanced <notifications@github.com> wrote:\n\n> Looks like ES and Timesketch mapping is still broken in 1.4 and 1.4.1\n>\n> elasticsearch.exceptions.RequestError: TransportError(400,\n> u'mapper_parsing_exception', u'Mapping definition for [_timestamp] has\n> unsupported parameters: [path : datetime]')\n>\n> —\n> You are receiving this because you modified the open/close state.\n> Reply to this email directly or view it on GitHub\n> <https://github.com/google/timesketch/issues/156#issuecomment-213767730>\n>\n"}}