{"action":"created","issue":{"url":"https://api.github.com/repos/uken/fluent-plugin-elasticsearch/issues/145","repository_url":"https://api.github.com/repos/uken/fluent-plugin-elasticsearch","labels_url":"https://api.github.com/repos/uken/fluent-plugin-elasticsearch/issues/145/labels{/name}","comments_url":"https://api.github.com/repos/uken/fluent-plugin-elasticsearch/issues/145/comments","events_url":"https://api.github.com/repos/uken/fluent-plugin-elasticsearch/issues/145/events","html_url":"https://github.com/uken/fluent-plugin-elasticsearch/issues/145","id":126124616,"number":145,"title":"Fix allowed characters in keys","user":{"login":"superdump","id":302146,"avatar_url":"https://avatars3.githubusercontent.com/u/302146?v=4","gravatar_id":"","url":"https://api.github.com/users/superdump","html_url":"https://github.com/superdump","followers_url":"https://api.github.com/users/superdump/followers","following_url":"https://api.github.com/users/superdump/following{/other_user}","gists_url":"https://api.github.com/users/superdump/gists{/gist_id}","starred_url":"https://api.github.com/users/superdump/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/superdump/subscriptions","organizations_url":"https://api.github.com/users/superdump/orgs","repos_url":"https://api.github.com/users/superdump/repos","events_url":"https://api.github.com/users/superdump/events{/privacy}","received_events_url":"https://api.github.com/users/superdump/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":4,"created_at":"2016-01-12T08:27:56Z","updated_at":"2018-03-08T00:52:34Z","closed_at":"2016-01-12T18:29:37Z","author_association":"NONE","body":"I am using the fluentd docker log driver and configuring it to add a bunch of labels that are applied to the container by swarm and others using reverse domain notation as used and recommended by docker.\n\nI am also using fluentd with this fluentd elasticsearch plugin. Unfortunately, elasticsearch does not like the '.' characters in the keys, so I get output like this:\n\n```\n[2016-01-12 08:12:05,215][DEBUG][action.bulk              ] [Punchout] [logstash-2016.01.12][3] failed to execute bulk item (index) index {[logstash-2016.01.12][fluentd][AVI05FLyAO15xpkHnohx], source[{\"com.example.site.applicationid\":\"logs\",\"com.example.site.componentid\":\"kibana\",\"container_id\":\"6b0e7cb8db11024bf712f0c56e29774d49b67cfaa5ca823bbf03569312b30245\",\"container_name\":\"/elated_williams\",\"source\":\"stdout\",\"log\":\"Some log text\",\"com.docker.swarm.id\":\"db0a0774ccdc4c131d1f69848b6390a55691f42a397f6fff7deb97f8d0ba293b\",\"@timestamp\":\"2016-01-12T08:11:26+00:00\"}]}\nMapperParsingException[Field name [com.docker.swarm.id] cannot contain '.']\n    at org.elasticsearch.index.mapper.object.ObjectMapper$TypeParser.parseProperties(ObjectMapper.java:278)\n    at org.elasticsearch.index.mapper.object.ObjectMapper$TypeParser.parseObjectOrDocumentTypeProperties(ObjectMapper.java:223)\n    at org.elasticsearch.index.mapper.object.RootObjectMapper$TypeParser.parse(RootObjectMapper.java:139)\n    at org.elasticsearch.index.mapper.DocumentMapperParser.parse(DocumentMapperParser.java:140)\n    at org.elasticsearch.index.mapper.DocumentMapperParser.parseCompressed(DocumentMapperParser.java:121)\n    at org.elasticsearch.index.mapper.MapperService.parse(MapperService.java:391)\n    at org.elasticsearch.cluster.metadata.MetaDataMappingService$2.execute(MetaDataMappingService.java:386)\n    at org.elasticsearch.cluster.service.InternalClusterService$UpdateTask.run(InternalClusterService.java:388)\n    at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:231)\n    at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:194)\n    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)\n    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)\n    at java.lang.Thread.run(Thread.java:745)\n```\n\nWould it be possible to re-write the keys in the plugin to ensure that they are elasticsearch-compatible? I tried to find some clear documentation of what characters are allowed in keys. I'll post here if I find anything useful.\n"},"comment":{"url":"https://api.github.com/repos/uken/fluent-plugin-elasticsearch/issues/comments/371339146","html_url":"https://github.com/uken/fluent-plugin-elasticsearch/issues/145#issuecomment-371339146","issue_url":"https://api.github.com/repos/uken/fluent-plugin-elasticsearch/issues/145","id":371339146,"user":{"login":"gtrevg","id":1063149,"avatar_url":"https://avatars1.githubusercontent.com/u/1063149?v=4","gravatar_id":"","url":"https://api.github.com/users/gtrevg","html_url":"https://github.com/gtrevg","followers_url":"https://api.github.com/users/gtrevg/followers","following_url":"https://api.github.com/users/gtrevg/following{/other_user}","gists_url":"https://api.github.com/users/gtrevg/gists{/gist_id}","starred_url":"https://api.github.com/users/gtrevg/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gtrevg/subscriptions","organizations_url":"https://api.github.com/users/gtrevg/orgs","repos_url":"https://api.github.com/users/gtrevg/repos","events_url":"https://api.github.com/users/gtrevg/events{/privacy}","received_events_url":"https://api.github.com/users/gtrevg/received_events","type":"User","site_admin":false},"created_at":"2018-03-08T00:52:34Z","updated_at":"2018-03-08T00:52:34Z","author_association":"NONE","body":"Just for reference, I wrote this a long while back which would handle the conversion:  https://github.com/gtrevg/fluent-plugin-dotted-keys"}}
{"actor":{"display_login":"gtrevg"}}