{"action":"created","issue":{"url":"https://api.github.com/repos/npgall/cqengine/issues/102","repository_url":"https://api.github.com/repos/npgall/cqengine","labels_url":"https://api.github.com/repos/npgall/cqengine/issues/102/labels{/name}","comments_url":"https://api.github.com/repos/npgall/cqengine/issues/102/comments","events_url":"https://api.github.com/repos/npgall/cqengine/issues/102/events","html_url":"https://github.com/npgall/cqengine/issues/102","id":189716526,"number":102,"title":"ExistsIn query performance","user":{"login":"uujava","id":5691016,"avatar_url":"https://avatars.githubusercontent.com/u/5691016?v=3","gravatar_id":"","url":"https://api.github.com/users/uujava","html_url":"https://github.com/uujava","followers_url":"https://api.github.com/users/uujava/followers","following_url":"https://api.github.com/users/uujava/following{/other_user}","gists_url":"https://api.github.com/users/uujava/gists{/gist_id}","starred_url":"https://api.github.com/users/uujava/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/uujava/subscriptions","organizations_url":"https://api.github.com/users/uujava/orgs","repos_url":"https://api.github.com/users/uujava/repos","events_url":"https://api.github.com/users/uujava/events{/privacy}","received_events_url":"https://api.github.com/users/uujava/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":4,"created_at":"2016-11-16T14:16:27Z","updated_at":"2016-11-18T15:29:03Z","closed_at":null,"body":"I have two collections with something like master-detail relation (ForeignKey -> PrimaryKey  in database terms). I basically need to perform, in most primitive form, a query like this:\r\n```\r\n Query<Item> query = QueryFactory.and(QueryFactory.equal(ID, randomId()), QueryFactory.existsIn(secondCollection, ID, EID));\r\nItem result = firstCollection.retrieve(query).iterator().next();\r\n```\r\nCollections are indexed by ID (UniqueIndex - like primary key) and EID (HashIndex - foreign key) respectively.\r\nI've put JMH benchmark to strip off my project specifics and compare different options of retrieving results: https://github.com/uujava/cqengine-query-jmh \r\nAnd got the results I could not explain: https://github.com/uujava/cqengine-query-jmh/releases/tag/initial_version\r\n\r\nComparing a case `QueryTest.streamJoin`\r\n```\r\n    @Benchmark\r\n    public void streamJoin(Blackhole bh) {\r\n        Long value = randomId();\r\n        Equal<Long, Long> equal = QueryFactory.equal(EID, value);\r\n        Long result = StreamSupport.stream(second.retrieve(equal).spliterator(), false).map(found -> {\r\n            Equal<Long, Long> subQ = QueryFactory.equal(ID, found);\r\n            return first.retrieve(subQ).iterator().next();\r\n        }).findFirst().get();\r\n        bh.consume(result);\r\n    }\r\n```\r\nWith a case `QueryTest.uniqueExistsJoin`\r\n```\r\n    @Benchmark\r\n    public void uniqueExistsJoin(Blackhole bh) {\r\n        Query<Long> query = QueryFactory.and(QueryFactory.equal(ID, randomId()), QueryFactory.existsIn(second, ID, EID));\r\n        bh.consume(first.retrieve(query).iterator().next());\r\n    }\r\n```\r\n\r\nI've got \r\n434088,196 vs 12,434 -> 35000 times difference.\r\n\r\njmh with -prof stack reveals that the `QueryTest.uniqueExistsJoin` most time spent in:\r\n```\r\n 55.2%  55.2% java.util.concurrent.ConcurrentHashMap$Traverser.advance\r\n 44.3%  44.3% java.util.AbstractSet.hashCode\r\n```\r\nI guess calculating hashCode on a big set of something.\r\n\r\nNot sure if it's an issue, or my misuse of existsIn API.\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/npgall/cqengine/issues/comments/261559387","html_url":"https://github.com/npgall/cqengine/issues/102#issuecomment-261559387","issue_url":"https://api.github.com/repos/npgall/cqengine/issues/102","id":261559387,"user":{"login":"uujava","id":5691016,"avatar_url":"https://avatars.githubusercontent.com/u/5691016?v=3","gravatar_id":"","url":"https://api.github.com/users/uujava","html_url":"https://github.com/uujava","followers_url":"https://api.github.com/users/uujava/followers","following_url":"https://api.github.com/users/uujava/following{/other_user}","gists_url":"https://api.github.com/users/uujava/gists{/gist_id}","starred_url":"https://api.github.com/users/uujava/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/uujava/subscriptions","organizations_url":"https://api.github.com/users/uujava/orgs","repos_url":"https://api.github.com/users/uujava/repos","events_url":"https://api.github.com/users/uujava/events{/privacy}","received_events_url":"https://api.github.com/users/uujava/received_events","type":"User","site_admin":false},"created_at":"2016-11-18T15:29:03Z","updated_at":"2016-11-18T15:29:03Z","body":"Please consider another performance pull request https://github.com/npgall/cqengine/pull/105 for review."}}
{"actor":{"display_login":"uujava"}}