{"action":"opened","issue":{"url":"https://api.github.com/repos/confluentinc/ksql/issues/1705","repository_url":"https://api.github.com/repos/confluentinc/ksql","labels_url":"https://api.github.com/repos/confluentinc/ksql/issues/1705/labels{/name}","comments_url":"https://api.github.com/repos/confluentinc/ksql/issues/1705/comments","events_url":"https://api.github.com/repos/confluentinc/ksql/issues/1705/events","html_url":"https://github.com/confluentinc/ksql/issues/1705","id":348218710,"node_id":"MDU6SXNzdWUzNDgyMTg3MTA=","number":1705,"title":"Better handling of serialisation errors","user":{"login":"rmoff","id":3671582,"node_id":"MDQ6VXNlcjM2NzE1ODI=","avatar_url":"https://avatars0.githubusercontent.com/u/3671582?v=4","gravatar_id":"","url":"https://api.github.com/users/rmoff","html_url":"https://github.com/rmoff","followers_url":"https://api.github.com/users/rmoff/followers","following_url":"https://api.github.com/users/rmoff/following{/other_user}","gists_url":"https://api.github.com/users/rmoff/gists{/gist_id}","starred_url":"https://api.github.com/users/rmoff/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rmoff/subscriptions","organizations_url":"https://api.github.com/users/rmoff/orgs","repos_url":"https://api.github.com/users/rmoff/repos","events_url":"https://api.github.com/users/rmoff/events{/privacy}","received_events_url":"https://api.github.com/users/rmoff/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-08-07T08:28:21Z","updated_at":"2018-08-07T08:28:21Z","closed_at":null,"author_association":"CONTRIBUTOR","body":"Currently KSQL will silently ignore serialisation errors (from the user's point of view - they are logged server side). \r\n\r\nI'm was getting no results from a `SELECT`, and it turned out the server log is full of `org.apache.kafka.common.errors.SerializationException`. IMO these should be floated to the user somehow. Otherwise they will puzzled why there is no data being returned. The message could be something as simple as: \r\n\r\n     WARN: some messages were skipped as they could not be deserialised. Please see <logfile> for details\r\n\r\n(we'd need to think about how that works w.r.t. a clustered deployment and pointing them at the relevant server). \r\n\r\nAnother option would be to have some variable threshold, below which specific serialisation errors are passed back to the console, and above which a catch-all is shown, for example: \r\n\r\n\r\n    WARN: Failed to deserialize message offset 1 partition 1 topic flood-monitoring-059793\r\n            KsqlJsonDeserializer : java.lang.ClassCastException: java.util.HashMap cannot be cast to java.util.List\r\n    WARN: Failed to deserialize message offset 2 partition 1 topic flood-monitoring-059793\r\n            KsqlJsonDeserializer : java.lang.ClassCastException: java.util.HashMap cannot be cast to java.util.List\r\n    WARN: Failed to deserialize message offset 3 partition 1 topic flood-monitoring-059793\r\n            KsqlJsonDeserializer : java.lang.ClassCastException: java.util.HashMap cannot be cast to java.util.List\r\n    WARN: More than 3 deserialisation errors occurred. Please check the server log file for full details.\r\n\r\nI would suggest passing back to the user a message, vs the other option of _just_ making the log more accessible. This is because if the user gets no results from a query, this could be for several reasons, and they shouldn't have to check the log \"just in case\" for serialisation errors. Reasons for no data include: \r\n  1. no data in the topic\r\n  2. offset is set to `latest`\r\n  3. serialisation error\r\n"}}
{"actor":{"display_login":"rmoff"}}