{"action":"created","issue":{"url":"https://api.github.com/repos/apache/arrow/issues/2681","repository_url":"https://api.github.com/repos/apache/arrow","labels_url":"https://api.github.com/repos/apache/arrow/issues/2681/labels{/name}","comments_url":"https://api.github.com/repos/apache/arrow/issues/2681/comments","events_url":"https://api.github.com/repos/apache/arrow/issues/2681/events","html_url":"https://github.com/apache/arrow/pull/2681","id":365570522,"node_id":"MDExOlB1bGxSZXF1ZXN0MjE5NDU5NjE1","number":2681,"title":"ARROW-3396: [WIP] [Java] VectorSchemaRoot.create(schema, allocator) should encoded vector for dictionary encoding","user":{"login":"icexelloss","id":793516,"node_id":"MDQ6VXNlcjc5MzUxNg==","avatar_url":"https://avatars0.githubusercontent.com/u/793516?v=4","gravatar_id":"","url":"https://api.github.com/users/icexelloss","html_url":"https://github.com/icexelloss","followers_url":"https://api.github.com/users/icexelloss/followers","following_url":"https://api.github.com/users/icexelloss/following{/other_user}","gists_url":"https://api.github.com/users/icexelloss/gists{/gist_id}","starred_url":"https://api.github.com/users/icexelloss/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/icexelloss/subscriptions","organizations_url":"https://api.github.com/users/icexelloss/orgs","repos_url":"https://api.github.com/users/icexelloss/repos","events_url":"https://api.github.com/users/icexelloss/events{/privacy}","received_events_url":"https://api.github.com/users/icexelloss/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":17,"created_at":"2018-10-01T17:46:24Z","updated_at":"2018-10-26T19:47:52Z","closed_at":null,"author_association":"CONTRIBUTOR","pull_request":{"url":"https://api.github.com/repos/apache/arrow/pulls/2681","html_url":"https://github.com/apache/arrow/pull/2681","diff_url":"https://github.com/apache/arrow/pull/2681.diff","patch_url":"https://github.com/apache/arrow/pull/2681.patch"},"body":"When creating a dictionary encoded vector from Schema, I expect that a vector of encoded type is created, however, currently will create a decoded vector, i.e.:\r\n\r\n```\r\n    // Create the dictionary\r\n    VarCharVector dictVector = (VarCharVector)\r\n        FieldType.nullable(new ArrowType.Utf8())\r\n            .createNewSingleVector(\"enum1_dict\", allocator, null);\r\n    dictVector.allocateNew();\r\n    dictVector.set(0, \"foo\".getBytes(StandardCharsets.UTF_8));\r\n    dictVector.set(1, \"bar\".getBytes(StandardCharsets.UTF_8));\r\n    dictVector.setValueCount(2);\r\n\r\n    DictionaryEncoding enum1Encoding = new DictionaryEncoding(\r\n        0,\r\n        true,\r\n        new ArrowType.Int(8, true)\r\n    );\r\n\r\n    DictionaryProvider dictionaryProvider = new DictionaryProvider.MapDictionaryProvider(\r\n        new Dictionary(dictVector, enum1Encoding)\r\n    );\r\n\r\n    // Create the dictionary encoded vector\r\n    Schema schema = new Schema(\r\n        asList(\r\n            new Field(\r\n                \"enum1\",\r\n                true,\r\n                // this is the type of the decoded value\r\n                new ArrowType.Utf8(),\r\n                enum1Encoding,\r\n                Collections.<Field>emptyList())\r\n    ));\r\n\r\n    // This doesn't not work without the patch\r\n    // root.getVector(\"enum1\") returns a VarCharVector instead of a TinyIntVector\r\n    VectorSchemaRoot root = VectorSchemaRoot.create(schema, allocator);\r\n    TinyIntVector vector = (TinyIntVector) root.getVector(\"enum1\");\r\n    vector.allocateNew();\r\n```\r\n\r\nI expect to get an encoded vector because that's the underlying memory representation. I also got an encoded vector when consuming a dictionary encoded vector so I think on the producer side it should be the same."},"comment":{"url":"https://api.github.com/repos/apache/arrow/issues/comments/433523217","html_url":"https://github.com/apache/arrow/pull/2681#issuecomment-433523217","issue_url":"https://api.github.com/repos/apache/arrow/issues/2681","id":433523217,"node_id":"MDEyOklzc3VlQ29tbWVudDQzMzUyMzIxNw==","user":{"login":"jacques-n","id":183350,"node_id":"MDQ6VXNlcjE4MzM1MA==","avatar_url":"https://avatars0.githubusercontent.com/u/183350?v=4","gravatar_id":"","url":"https://api.github.com/users/jacques-n","html_url":"https://github.com/jacques-n","followers_url":"https://api.github.com/users/jacques-n/followers","following_url":"https://api.github.com/users/jacques-n/following{/other_user}","gists_url":"https://api.github.com/users/jacques-n/gists{/gist_id}","starred_url":"https://api.github.com/users/jacques-n/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jacques-n/subscriptions","organizations_url":"https://api.github.com/users/jacques-n/orgs","repos_url":"https://api.github.com/users/jacques-n/repos","events_url":"https://api.github.com/users/jacques-n/events{/privacy}","received_events_url":"https://api.github.com/users/jacques-n/received_events","type":"User","site_admin":false},"created_at":"2018-10-26T19:47:52Z","updated_at":"2018-10-26T19:47:52Z","author_association":"CONTRIBUTOR","body":"I find this entire thread quite hard to follow. It feels like we're developing APIs without a clear set of sample usages in mind. Can we start with the example program that we want to use the APIs and discuss the key requirements? For example:\r\n\r\n- Some consumers want to interact with encoded values.\r\n- Some consumers want to interact with decoded values.\r\n- There should be tools to decorate or move from encoding to decoded values and vice versa.\r\n- The patterns for each should preferably look like ... and ...\r\n- etc...\r\n\r\nPer JSON comments, yes, Jackson should be an optional separate module around json serialization. I would think it is entirely its own module and doesn't relate to other tools, etc.\r\n"}}
{"actor":{"display_login":"jacques-n"}}