{"action":"created","issue":{"url":"https://api.github.com/repos/hpgrahsl/kafka-connect-mongodb/issues/56","repository_url":"https://api.github.com/repos/hpgrahsl/kafka-connect-mongodb","labels_url":"https://api.github.com/repos/hpgrahsl/kafka-connect-mongodb/issues/56/labels{/name}","comments_url":"https://api.github.com/repos/hpgrahsl/kafka-connect-mongodb/issues/56/comments","events_url":"https://api.github.com/repos/hpgrahsl/kafka-connect-mongodb/issues/56/events","html_url":"https://github.com/hpgrahsl/kafka-connect-mongodb/issues/56","id":372203474,"node_id":"MDU6SXNzdWUzNzIyMDM0NzQ=","number":56,"title":"Sinking aggregated data does not work","user":{"login":"masihyeganeh","id":692335,"node_id":"MDQ6VXNlcjY5MjMzNQ==","avatar_url":"https://avatars1.githubusercontent.com/u/692335?v=4","gravatar_id":"","url":"https://api.github.com/users/masihyeganeh","html_url":"https://github.com/masihyeganeh","followers_url":"https://api.github.com/users/masihyeganeh/followers","following_url":"https://api.github.com/users/masihyeganeh/following{/other_user}","gists_url":"https://api.github.com/users/masihyeganeh/gists{/gist_id}","starred_url":"https://api.github.com/users/masihyeganeh/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/masihyeganeh/subscriptions","organizations_url":"https://api.github.com/users/masihyeganeh/orgs","repos_url":"https://api.github.com/users/masihyeganeh/repos","events_url":"https://api.github.com/users/masihyeganeh/events{/privacy}","received_events_url":"https://api.github.com/users/masihyeganeh/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-10-20T11:49:32Z","updated_at":"2018-10-20T12:55:42Z","closed_at":null,"author_association":"NONE","body":"Hi. I have an issue with Sinking data to MongoDB using your code.\r\nI read similar issues like #55 and #36 and I know possible solutions but I couldn't make them work with my situation.\r\nI have a stream that I aggregate values of it within a window using KSQL. here is my query:\r\n\r\n```sql\r\nCREATE TABLE WATCHTIME_AGG AS \\\r\n SELECT ID, MAX(progress) AS progress, MAX(position) AS position, MAX(timestamp) AS timestamp \\\r\n   FROM WATCHTIME \\\r\n   WINDOW TUMBLING (SIZE 1 HOUR) \\\r\n   GROUP BY ID;\r\n```\r\n\r\nIt creates a table which it's ROWKEYs are like this:\r\n\r\n- `149988257825337_6696933 : Window{start=1540024620000 end=-}`\r\n- `null_11646895 : Window{start=1540024620000 end=-}`\r\n- `153974245826038_6222197 : Window{start=1540024620000 end=-}`\r\n- `151415963389431_11197398 : Window{start=1540024620000 end=-}`\r\n- `153173851492214_9207426 : Window{start=1540024620000 end=-}`\r\n\r\nBut it can't be Sinked because of JSON error when key type is JSON:\r\n`com.fasterxml.jackson.core.JsonParseException: Invalid UTF-8 start byte 0x90\\n at [Source: (byte[])\\\"null_12208937\\u0000\\u0000\\u0001f\\ufffd\\u0007\\ufffd\\u0000\\\"; line: 1, column: 19]`\r\n\r\nAnd this error when key type is String:\r\n`org.bson.json.JsonParseException: JSON reader was expecting a value but found 'null_12208937'`\r\n\r\nThe key should be a valid JSON or Avro if I'm not wrong, but window aggregation won't produce JSON and won't let me change it to a valid one.\r\n\r\nWhat can I do to make it work?\r\nIt's a really simple use-case and should be straightforward. But at least I could not make it work."},"comment":{"url":"https://api.github.com/repos/hpgrahsl/kafka-connect-mongodb/issues/comments/431578338","html_url":"https://github.com/hpgrahsl/kafka-connect-mongodb/issues/56#issuecomment-431578338","issue_url":"https://api.github.com/repos/hpgrahsl/kafka-connect-mongodb/issues/56","id":431578338,"node_id":"MDEyOklzc3VlQ29tbWVudDQzMTU3ODMzOA==","user":{"login":"masihyeganeh","id":692335,"node_id":"MDQ6VXNlcjY5MjMzNQ==","avatar_url":"https://avatars1.githubusercontent.com/u/692335?v=4","gravatar_id":"","url":"https://api.github.com/users/masihyeganeh","html_url":"https://github.com/masihyeganeh","followers_url":"https://api.github.com/users/masihyeganeh/followers","following_url":"https://api.github.com/users/masihyeganeh/following{/other_user}","gists_url":"https://api.github.com/users/masihyeganeh/gists{/gist_id}","starred_url":"https://api.github.com/users/masihyeganeh/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/masihyeganeh/subscriptions","organizations_url":"https://api.github.com/users/masihyeganeh/orgs","repos_url":"https://api.github.com/users/masihyeganeh/repos","events_url":"https://api.github.com/users/masihyeganeh/events{/privacy}","received_events_url":"https://api.github.com/users/masihyeganeh/received_events","type":"User","site_admin":false},"created_at":"2018-10-20T12:55:42Z","updated_at":"2018-10-20T12:55:42Z","author_association":"NONE","body":"I also tried hoisting with no success.\r\n```json\r\n{\r\n  \"connector.class\": \"at.grahsl.kafka.connect.mongodb.MongoDbSinkConnector\",\r\n  \"topics\": \"WATCHTIME_AGG\",\r\n  \"mongodb.connection.uri\": \"mongodb://...?w=1&journal=true\",\r\n  \"name\": \"watchtime-mongo\",\r\n  \"value.converter.schema.registry.url\": \"http://schema-registry:8081\",\r\n  \"value.converter.schemas.enable\": \"false\",\r\n  \"value.converter\": \"org.apache.kafka.connect.json.JsonConverter\",\r\n  \"key.converter.schema.registry.url\": \"http://schema-registry:8081\",\r\n  \"key.converter.schemas.enable\": \"false\",\r\n  \"key.converter\": \"org.apache.kafka.connect.json.JsonConverter\",\r\n  \"mongodb.collection\": \"watchtime\",\r\n  \"transforms\": \"id_to_object\",\r\n  \"transforms.id_to_object.type\": \"org.apache.kafka.connect.transforms.HoistField$Key\",\r\n  \"transforms.id_to_object.field\": \"id\"\r\n}\r\n```"}}
{"actor":{"display_login":"masihyeganeh"}}