{"action":"created","issue":{"url":"https://api.github.com/repos/prestodb/presto/issues/9975","repository_url":"https://api.github.com/repos/prestodb/presto","labels_url":"https://api.github.com/repos/prestodb/presto/issues/9975/labels{/name}","comments_url":"https://api.github.com/repos/prestodb/presto/issues/9975/comments","events_url":"https://api.github.com/repos/prestodb/presto/issues/9975/events","html_url":"https://github.com/prestodb/presto/pull/9975","id":297579923,"number":9975,"title":"Fix type mismatch in Parquet predicate pushdown","user":{"login":"nishantrayan","id":312902,"avatar_url":"https://avatars1.githubusercontent.com/u/312902?v=4","gravatar_id":"","url":"https://api.github.com/users/nishantrayan","html_url":"https://github.com/nishantrayan","followers_url":"https://api.github.com/users/nishantrayan/followers","following_url":"https://api.github.com/users/nishantrayan/following{/other_user}","gists_url":"https://api.github.com/users/nishantrayan/gists{/gist_id}","starred_url":"https://api.github.com/users/nishantrayan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nishantrayan/subscriptions","organizations_url":"https://api.github.com/users/nishantrayan/orgs","repos_url":"https://api.github.com/users/nishantrayan/repos","events_url":"https://api.github.com/users/nishantrayan/events{/privacy}","received_events_url":"https://api.github.com/users/nishantrayan/received_events","type":"User","site_admin":false},"labels":[{"id":196992743,"url":"https://api.github.com/repos/prestodb/presto/labels/CLA%20Signed","name":"CLA Signed","color":"ededed","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2018-02-15T20:08:03Z","updated_at":"2018-02-27T07:45:53Z","closed_at":null,"author_association":"NONE","pull_request":{"url":"https://api.github.com/repos/prestodb/presto/pulls/9975","html_url":"https://github.com/prestodb/presto/pull/9975","diff_url":"https://github.com/prestodb/presto/pull/9975.diff","patch_url":"https://github.com/prestodb/presto/pull/9975.patch"},"body":"The parquet predicate pushdown logic was refactored to allow nested pushdown in the beginning of last year and seems like there was one minor but annoying bug that is still in the latest version and issue was opened here: https://github.com/prestodb/presto/issues/9084\r\nWe use presto with hive and can reliably reproduce this with the schema containing column of fixed varchar (say varchar(255))\r\nThe effectivePredicate seems to have the hive's column type which is varchar(255) however the type header of the parquet chunk gets translated to varchar and because of this mismatch we fail the type check that happens later.\r\n\r\n```\r\nCaused by: java.lang.IllegalArgumentException: Mismatched Domain types: varchar(255) vs varchar\r\n\tat com.facebook.presto.spi.predicate.Domain.checkCompatibility(Domain.java:228)\r\n\tat com.facebook.presto.spi.predicate.Domain.intersect(Domain.java:183)\r\n\tat com.facebook.presto.spi.predicate.TupleDomain.intersect(TupleDomain.java:196)\r\n\tat com.facebook.presto.spi.predicate.TupleDomain.overlaps(TupleDomain.java:299)\r\n```\r\n\r\nPlease have a look at this @nezihyigitbasi @zhenxiao @dain  "},"comment":{"url":"https://api.github.com/repos/prestodb/presto/issues/comments/368775885","html_url":"https://github.com/prestodb/presto/pull/9975#issuecomment-368775885","issue_url":"https://api.github.com/repos/prestodb/presto/issues/9975","id":368775885,"user":{"login":"nezihyigitbasi","id":1223839,"avatar_url":"https://avatars3.githubusercontent.com/u/1223839?v=4","gravatar_id":"","url":"https://api.github.com/users/nezihyigitbasi","html_url":"https://github.com/nezihyigitbasi","followers_url":"https://api.github.com/users/nezihyigitbasi/followers","following_url":"https://api.github.com/users/nezihyigitbasi/following{/other_user}","gists_url":"https://api.github.com/users/nezihyigitbasi/gists{/gist_id}","starred_url":"https://api.github.com/users/nezihyigitbasi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nezihyigitbasi/subscriptions","organizations_url":"https://api.github.com/users/nezihyigitbasi/orgs","repos_url":"https://api.github.com/users/nezihyigitbasi/repos","events_url":"https://api.github.com/users/nezihyigitbasi/events{/privacy}","received_events_url":"https://api.github.com/users/nezihyigitbasi/received_events","type":"User","site_admin":false},"created_at":"2018-02-27T07:45:53Z","updated_at":"2018-02-27T07:45:53Z","author_association":"CONTRIBUTOR","body":"I feel like this is a bit hacky as we get the type of the column from the predicate instead of the column descriptor (which currently does not have that information). So, I think a more proper approach is to add the Presto/Hive type info to `RichColumnDescriptor`. \r\n\r\nIn`ParquetPageSourceFactory::createParquetPageSource()` we have the `HiveType`s as part of `List<HiveColumnHandle> columns` (from `HiveType` you can get the type information you want) and we can keep this type information in the `RichColumnDescriptor` when we create them. Then in `ParquetTypeUtils::getPrestoType()` we can consult that type info to get the Presto types. What do you think?"}}
{"actor":{"display_login":"nezihyigitbasi"}}