{"action":"created","issue":{"url":"https://api.github.com/repos/shun0102/embulk-parser-jsonl/issues/9","repository_url":"https://api.github.com/repos/shun0102/embulk-parser-jsonl","labels_url":"https://api.github.com/repos/shun0102/embulk-parser-jsonl/issues/9/labels{/name}","comments_url":"https://api.github.com/repos/shun0102/embulk-parser-jsonl/issues/9/comments","events_url":"https://api.github.com/repos/shun0102/embulk-parser-jsonl/issues/9/events","html_url":"https://github.com/shun0102/embulk-parser-jsonl/pull/9","id":154967415,"number":9,"title":"Support typecast","user":{"login":"sonots","id":2290461,"avatar_url":"https://avatars.githubusercontent.com/u/2290461?v=3","gravatar_id":"","url":"https://api.github.com/users/sonots","html_url":"https://github.com/sonots","followers_url":"https://api.github.com/users/sonots/followers","following_url":"https://api.github.com/users/sonots/following{/other_user}","gists_url":"https://api.github.com/users/sonots/gists{/gist_id}","starred_url":"https://api.github.com/users/sonots/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sonots/subscriptions","organizations_url":"https://api.github.com/users/sonots/orgs","repos_url":"https://api.github.com/users/sonots/repos","events_url":"https://api.github.com/users/sonots/events{/privacy}","received_events_url":"https://api.github.com/users/sonots/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":5,"created_at":"2016-05-16T06:14:04Z","updated_at":"2016-05-27T01:37:18Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/shun0102/embulk-parser-jsonl/pulls/9","html_url":"https://github.com/shun0102/embulk-parser-jsonl/pull/9","diff_url":"https://github.com/shun0102/embulk-parser-jsonl/pull/9.diff","patch_url":"https://github.com/shun0102/embulk-parser-jsonl/pull/9.patch"},"body":"Added `default_typecast` and `typecast` options to support type casting from various types to various types. `column_options` which supported only from String to boolean, long, double was deprecated.\r\n\r\nTo keep compatibility with lower versions, the default_typecast is true as default, and `column_options` are still available with a deprecation warning message.\r\n\r\n```\r\nsrc/main/java/org/embulk/parser/\r\n└── jsonl\r\n    ├── ColumnCaster.java\r\n    ├── ColumnVisitorImpl.java\r\n    ├── JsonRecordValidateException.java\r\n    ├── JsonlParserPlugin.java\r\n    └── cast\r\n        ├── BooleanCast.java\r\n        ├── DoubleCast.java\r\n        ├── JsonCast.java\r\n        ├── LongCast.java\r\n        └── StringCast.java\r\n```\r\n\r\nSupporting typecast is useful in various cases although JSON itself has type of data such as boolean, integer, float, string. For example, Exporting BigQuery table with a JSON format dumps FLOAT column into integer (if decimal points are 0) or float (if decimal points are not 0). That is, types would be varying in each line.\r\n\r\nTo read such JSON text, we need to cast integer to float, and float to float (that is, no cast) automatically. This PR enables to do such parsing.\r\n\r\nEDIT: I tried to make yaml configuration interface to be similar with parser-csv.\r\nIn parser csv, we write embulk types to cast in `columns` option (although these casts are all from String).\r\nNew interface allows us to write embulk types to cast in `columns` option similarly.\r\n\r\n```\r\nin:\r\n  type: file\r\n  path_prefix: \"example/sample\"\r\n  parser:\r\n    type: jsonl\r\n    # default_typecast: true # default: true\r\n    columns:\r\n      - {name: \"foo\", type: \"string\"}\r\n      - {name: \"bool\", type: \"boolean\"}\r\n      - {name: \"bool_str\", type: \"boolean\"}\r\n      - {name: \"int\", type: \"long\"}\r\n      - {name: \"int_str\", type: \"long\"}\r\n      - {name: \"time\", type: \"timestamp\", format: '%Y-%m-%d %H:%M:%S'}\r\n      - {name: \"double\", type: \"double\"}\r\n      - {name: \"double_str\", type: \"double\"}\r\n      - {name: \"array\", type: \"json\"}\r\n    stop_on_invalid_record: true\r\nout:\r\n  type: stdout\r\n```"},"comment":{"url":"https://api.github.com/repos/shun0102/embulk-parser-jsonl/issues/comments/222039614","html_url":"https://github.com/shun0102/embulk-parser-jsonl/pull/9#issuecomment-222039614","issue_url":"https://api.github.com/repos/shun0102/embulk-parser-jsonl/issues/9","id":222039614,"user":{"login":"muga","id":127959,"avatar_url":"https://avatars.githubusercontent.com/u/127959?v=3","gravatar_id":"","url":"https://api.github.com/users/muga","html_url":"https://github.com/muga","followers_url":"https://api.github.com/users/muga/followers","following_url":"https://api.github.com/users/muga/following{/other_user}","gists_url":"https://api.github.com/users/muga/gists{/gist_id}","starred_url":"https://api.github.com/users/muga/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/muga/subscriptions","organizations_url":"https://api.github.com/users/muga/orgs","repos_url":"https://api.github.com/users/muga/repos","events_url":"https://api.github.com/users/muga/events{/privacy}","received_events_url":"https://api.github.com/users/muga/received_events","type":"User","site_admin":false},"created_at":"2016-05-27T01:37:18Z","updated_at":"2016-05-27T01:37:18Z","body":"@sonots I roughly saw your PR. I agree with your config design and will accept and merge it. I have 2 comments now. \r\n\r\na. Can you add unit tests for your change? You can see the test coverage rate by `$ ./gradlew clean test jacocoTestReport`.\r\n\r\nb. I'm worry about the performance degradation cased by `if` statements that the type castings are using. Do you have any insight?\r\n\r\nI will take a time later to read your PR more deeply and let you know if I have some questions."}}
{"actor":{"display_login":"muga"}}