{"action":"created","comment":{"url":"https://api.github.com/repos/Eshcar/hbase/pulls/comments/59506418","id":59506418,"diff_hunk":"@@ -43,4 +112,144 @@ public KeyValueScanner getKeyValueScanner() {\n     return new CollectionBackedScanner(getCellSet(), getComparator());\n   }\n \n+  /**------------------------------------------------------------------------\n+   * Change the CellSet of this ImmutableSegment from one based on ConcurrentSkipListMap to one\n+   * based on CellArrayMap.\n+   * If this ImmutableSegment is not based on ConcurrentSkipListMap , this is NOP\n+   * For now the change from ConcurrentSkipListMap to CellChunkMap is not supported, because\n+   * this requires the Cell to know on which Chunk it is placed.\n+   *\n+   * Synchronization of the CellSet replacement:\n+   * The reference to the CellSet is AtomicReference and is updated only when ImmutableSegment\n+   * is constructed (single thread) or flattened. The flattening happens as part of a single\n+   * thread of compaction, but to be on the safe side the initial CellSet is locally saved\n+   * before the flattening and then replaced using CAS instruction.\n+   */\n+  public boolean flatten() {\n+    if (isFlat) return false;\n+    CellSet oldCellSet = getCellSet();\n+    int numOfCells = getCellsCount();\n+\n+    // each Cell is now represented in either in CellArrayMap or in CellChunkMap\n+    constantCellMetaDataSize = ClassSize.CELL_ARRAY_MAP_ENTRY;\n+\n+    // arrange the meta-data size, decrease all meta-data sizes related to SkipList\n+    incSize(\n+        -(ClassSize.CONCURRENT_SKIPLISTMAP + numOfCells * ClassSize.CONCURRENT_SKIPLISTMAP_ENTRY));\n+    // add size of CellArrayMap and meta-data overhead per Cell\n+    incSize(ClassSize.CELL_CHUNK_MAP + numOfCells * constantCellMetaDataSize);\n+\n+    CellSet  newCellSet = recreateCellArrayMapSet(numOfCells); // build the CellSet CellArrayMap based\n+    setCellSet(oldCellSet,newCellSet);\n+    return true;\n+  }\n+\n+  /////////////////////  PRIVATE METHODS  /////////////////////\n+  /*------------------------------------------------------------------------*/\n+  // Create CellSet based on CellArrayMap from compacting iterator\n+  private CellSet createCellArrayMapSet(int numOfCells, MemStoreCompactorIterator iterator) {","path":"hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ImmutableSegment.java","position":128,"original_position":128,"commit_id":"d2a9a20b70c96ecd20770aa9fff11c2b29b7b7da","original_commit_id":"d2a9a20b70c96ecd20770aa9fff11c2b29b7b7da","user":{"login":"ebortnik","id":5113610,"avatar_url":"https://avatars.githubusercontent.com/u/5113610?v=3","gravatar_id":"","url":"https://api.github.com/users/ebortnik","html_url":"https://github.com/ebortnik","followers_url":"https://api.github.com/users/ebortnik/followers","following_url":"https://api.github.com/users/ebortnik/following{/other_user}","gists_url":"https://api.github.com/users/ebortnik/gists{/gist_id}","starred_url":"https://api.github.com/users/ebortnik/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ebortnik/subscriptions","organizations_url":"https://api.github.com/users/ebortnik/orgs","repos_url":"https://api.github.com/users/ebortnik/repos","events_url":"https://api.github.com/users/ebortnik/events{/privacy}","received_events_url":"https://api.github.com/users/ebortnik/received_events","type":"User","site_admin":false},"body":"Okay, getting the idea now. Please don't get mad, but you might need one more step, maybe not in this patch. \r\n\r\nIn this implementation, ImmutableSegment is actually a factory. Most of the logic is around creation/transformation of CellSet's. It makes sense to me to subclass ImmutableSegment into the following classes: (1) ISWrapper, (2) ISCellObjectArray, and (3) ISChunkArray. Then, put a factory on top of them - to be invoked by the Compactor. \r\nTake a look at the current code - there is near-zero overlap between these code paths - calling for polymorphism.  \r\n\r\nAt least, please document this design point. ","created_at":"2016-04-13T07:53:25Z","updated_at":"2016-04-13T07:53:25Z","html_url":"https://github.com/Eshcar/hbase/pull/6#discussion_r59506418","pull_request_url":"https://api.github.com/repos/Eshcar/hbase/pulls/6","_links":{"self":{"href":"https://api.github.com/repos/Eshcar/hbase/pulls/comments/59506418"},"html":{"href":"https://github.com/Eshcar/hbase/pull/6#discussion_r59506418"},"pull_request":{"href":"https://api.github.com/repos/Eshcar/hbase/pulls/6"}}},"pull_request":{"url":"https://api.github.com/repos/Eshcar/hbase/pulls/6","id":65467921,"html_url":"https://github.com/Eshcar/hbase/pull/6","diff_url":"https://github.com/Eshcar/hbase/pull/6.diff","patch_url":"https://github.com/Eshcar/hbase/pull/6.patch","issue_url":"https://api.github.com/repos/Eshcar/hbase/issues/6","number":6,"state":"open","locked":false,"title":"Hbase 14921 v3 - please review the recent addition of the CellFlatMap to CompactingMemStore","user":{"login":"sanastas","id":12136181,"avatar_url":"https://avatars.githubusercontent.com/u/12136181?v=3","gravatar_id":"","url":"https://api.github.com/users/sanastas","html_url":"https://github.com/sanastas","followers_url":"https://api.github.com/users/sanastas/followers","following_url":"https://api.github.com/users/sanastas/following{/other_user}","gists_url":"https://api.github.com/users/sanastas/gists{/gist_id}","starred_url":"https://api.github.com/users/sanastas/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sanastas/subscriptions","organizations_url":"https://api.github.com/users/sanastas/orgs","repos_url":"https://api.github.com/users/sanastas/repos","events_url":"https://api.github.com/users/sanastas/events{/privacy}","received_events_url":"https://api.github.com/users/sanastas/received_events","type":"User","site_admin":false},"body":"@Eshcar @ebortnik\r\n\r\nAs you know the code is not yet finished, but I would like to here your comments on the current state...","created_at":"2016-04-06T13:42:07Z","updated_at":"2016-04-13T07:53:25Z","closed_at":null,"merged_at":null,"merge_commit_sha":"f7d31cefee53dbfcff4d9016b69e00a6df692fef","assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/Eshcar/hbase/pulls/6/commits","review_comments_url":"https://api.github.com/repos/Eshcar/hbase/pulls/6/comments","review_comment_url":"https://api.github.com/repos/Eshcar/hbase/pulls/comments{/number}","comments_url":"https://api.github.com/repos/Eshcar/hbase/issues/6/comments","statuses_url":"https://api.github.com/repos/Eshcar/hbase/statuses/d2a9a20b70c96ecd20770aa9fff11c2b29b7b7da","head":{"label":"Eshcar:HBASE-14921-V3","ref":"HBASE-14921-V3","sha":"d2a9a20b70c96ecd20770aa9fff11c2b29b7b7da","user":{"login":"Eshcar","id":4384969,"avatar_url":"https://avatars.githubusercontent.com/u/4384969?v=3","gravatar_id":"","url":"https://api.github.com/users/Eshcar","html_url":"https://github.com/Eshcar","followers_url":"https://api.github.com/users/Eshcar/followers","following_url":"https://api.github.com/users/Eshcar/following{/other_user}","gists_url":"https://api.github.com/users/Eshcar/gists{/gist_id}","starred_url":"https://api.github.com/users/Eshcar/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Eshcar/subscriptions","organizations_url":"https://api.github.com/users/Eshcar/orgs","repos_url":"https://api.github.com/users/Eshcar/repos","events_url":"https://api.github.com/users/Eshcar/events{/privacy}","received_events_url":"https://api.github.com/users/Eshcar/received_events","type":"User","site_admin":false},"repo":{"id":30583208,"name":"hbase","full_name":"Eshcar/hbase","owner":{"login":"Eshcar","id":4384969,"avatar_url":"https://avatars.githubusercontent.com/u/4384969?v=3","gravatar_id":"","url":"https://api.github.com/users/Eshcar","html_url":"https://github.com/Eshcar","followers_url":"https://api.github.com/users/Eshcar/followers","following_url":"https://api.github.com/users/Eshcar/following{/other_user}","gists_url":"https://api.github.com/users/Eshcar/gists{/gist_id}","starred_url":"https://api.github.com/users/Eshcar/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Eshcar/subscriptions","organizations_url":"https://api.github.com/users/Eshcar/orgs","repos_url":"https://api.github.com/users/Eshcar/repos","events_url":"https://api.github.com/users/Eshcar/events{/privacy}","received_events_url":"https://api.github.com/users/Eshcar/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/Eshcar/hbase","description":"Mirror of Apache HBase","fork":true,"url":"https://api.github.com/repos/Eshcar/hbase","forks_url":"https://api.github.com/repos/Eshcar/hbase/forks","keys_url":"https://api.github.com/repos/Eshcar/hbase/keys{/key_id}","collaborators_url":"https://api.github.com/repos/Eshcar/hbase/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/Eshcar/hbase/teams","hooks_url":"https://api.github.com/repos/Eshcar/hbase/hooks","issue_events_url":"https://api.github.com/repos/Eshcar/hbase/issues/events{/number}","events_url":"https://api.github.com/repos/Eshcar/hbase/events","assignees_url":"https://api.github.com/repos/Eshcar/hbase/assignees{/user}","branches_url":"https://api.github.com/repos/Eshcar/hbase/branches{/branch}","tags_url":"https://api.github.com/repos/Eshcar/hbase/tags","blobs_url":"https://api.github.com/repos/Eshcar/hbase/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/Eshcar/hbase/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/Eshcar/hbase/git/refs{/sha}","trees_url":"https://api.github.com/repos/Eshcar/hbase/git/trees{/sha}","statuses_url":"https://api.github.com/repos/Eshcar/hbase/statuses/{sha}","languages_url":"https://api.github.com/repos/Eshcar/hbase/languages","stargazers_url":"https://api.github.com/repos/Eshcar/hbase/stargazers","contributors_url":"https://api.github.com/repos/Eshcar/hbase/contributors","subscribers_url":"https://api.github.com/repos/Eshcar/hbase/subscribers","subscription_url":"https://api.github.com/repos/Eshcar/hbase/subscription","commits_url":"https://api.github.com/repos/Eshcar/hbase/commits{/sha}","git_commits_url":"https://api.github.com/repos/Eshcar/hbase/git/commits{/sha}","comments_url":"https://api.github.com/repos/Eshcar/hbase/comments{/number}","issue_comment_url":"https://api.github.com/repos/Eshcar/hbase/issues/comments{/number}","contents_url":"https://api.github.com/repos/Eshcar/hbase/contents/{+path}","compare_url":"https://api.github.com/repos/Eshcar/hbase/compare/{base}...{head}","merges_url":"https://api.github.com/repos/Eshcar/hbase/merges","archive_url":"https://api.github.com/repos/Eshcar/hbase/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/Eshcar/hbase/downloads","issues_url":"https://api.github.com/repos/Eshcar/hbase/issues{/number}","pulls_url":"https://api.github.com/repos/Eshcar/hbase/pulls{/number}","milestones_url":"https://api.github.com/repos/Eshcar/hbase/milestones{/number}","notifications_url":"https://api.github.com/repos/Eshcar/hbase/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/Eshcar/hbase/labels{/name}","releases_url":"https://api.github.com/repos/Eshcar/hbase/releases{/id}","deployments_url":"https://api.github.com/repos/Eshcar/hbase/deployments","created_at":"2015-02-10T08:50:25Z","updated_at":"2016-01-17T10:18:51Z","pushed_at":"2016-04-13T07:15:04Z","git_url":"git://github.com/Eshcar/hbase.git","ssh_url":"git@github.com:Eshcar/hbase.git","clone_url":"https://github.com/Eshcar/hbase.git","svn_url":"https://github.com/Eshcar/hbase","homepage":null,"size":217094,"stargazers_count":0,"watchers_count":0,"language":"Java","has_issues":false,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":1,"mirror_url":null,"open_issues_count":5,"forks":1,"open_issues":5,"watchers":0,"default_branch":"master"}},"base":{"label":"Eshcar:HBASE-14920","ref":"HBASE-14920","sha":"648abf44773b9fad50fab48f4bea136502223b98","user":{"login":"Eshcar","id":4384969,"avatar_url":"https://avatars.githubusercontent.com/u/4384969?v=3","gravatar_id":"","url":"https://api.github.com/users/Eshcar","html_url":"https://github.com/Eshcar","followers_url":"https://api.github.com/users/Eshcar/followers","following_url":"https://api.github.com/users/Eshcar/following{/other_user}","gists_url":"https://api.github.com/users/Eshcar/gists{/gist_id}","starred_url":"https://api.github.com/users/Eshcar/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Eshcar/subscriptions","organizations_url":"https://api.github.com/users/Eshcar/orgs","repos_url":"https://api.github.com/users/Eshcar/repos","events_url":"https://api.github.com/users/Eshcar/events{/privacy}","received_events_url":"https://api.github.com/users/Eshcar/received_events","type":"User","site_admin":false},"repo":{"id":30583208,"name":"hbase","full_name":"Eshcar/hbase","owner":{"login":"Eshcar","id":4384969,"avatar_url":"https://avatars.githubusercontent.com/u/4384969?v=3","gravatar_id":"","url":"https://api.github.com/users/Eshcar","html_url":"https://github.com/Eshcar","followers_url":"https://api.github.com/users/Eshcar/followers","following_url":"https://api.github.com/users/Eshcar/following{/other_user}","gists_url":"https://api.github.com/users/Eshcar/gists{/gist_id}","starred_url":"https://api.github.com/users/Eshcar/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Eshcar/subscriptions","organizations_url":"https://api.github.com/users/Eshcar/orgs","repos_url":"https://api.github.com/users/Eshcar/repos","events_url":"https://api.github.com/users/Eshcar/events{/privacy}","received_events_url":"https://api.github.com/users/Eshcar/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/Eshcar/hbase","description":"Mirror of Apache HBase","fork":true,"url":"https://api.github.com/repos/Eshcar/hbase","forks_url":"https://api.github.com/repos/Eshcar/hbase/forks","keys_url":"https://api.github.com/repos/Eshcar/hbase/keys{/key_id}","collaborators_url":"https://api.github.com/repos/Eshcar/hbase/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/Eshcar/hbase/teams","hooks_url":"https://api.github.com/repos/Eshcar/hbase/hooks","issue_events_url":"https://api.github.com/repos/Eshcar/hbase/issues/events{/number}","events_url":"https://api.github.com/repos/Eshcar/hbase/events","assignees_url":"https://api.github.com/repos/Eshcar/hbase/assignees{/user}","branches_url":"https://api.github.com/repos/Eshcar/hbase/branches{/branch}","tags_url":"https://api.github.com/repos/Eshcar/hbase/tags","blobs_url":"https://api.github.com/repos/Eshcar/hbase/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/Eshcar/hbase/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/Eshcar/hbase/git/refs{/sha}","trees_url":"https://api.github.com/repos/Eshcar/hbase/git/trees{/sha}","statuses_url":"https://api.github.com/repos/Eshcar/hbase/statuses/{sha}","languages_url":"https://api.github.com/repos/Eshcar/hbase/languages","stargazers_url":"https://api.github.com/repos/Eshcar/hbase/stargazers","contributors_url":"https://api.github.com/repos/Eshcar/hbase/contributors","subscribers_url":"https://api.github.com/repos/Eshcar/hbase/subscribers","subscription_url":"https://api.github.com/repos/Eshcar/hbase/subscription","commits_url":"https://api.github.com/repos/Eshcar/hbase/commits{/sha}","git_commits_url":"https://api.github.com/repos/Eshcar/hbase/git/commits{/sha}","comments_url":"https://api.github.com/repos/Eshcar/hbase/comments{/number}","issue_comment_url":"https://api.github.com/repos/Eshcar/hbase/issues/comments{/number}","contents_url":"https://api.github.com/repos/Eshcar/hbase/contents/{+path}","compare_url":"https://api.github.com/repos/Eshcar/hbase/compare/{base}...{head}","merges_url":"https://api.github.com/repos/Eshcar/hbase/merges","archive_url":"https://api.github.com/repos/Eshcar/hbase/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/Eshcar/hbase/downloads","issues_url":"https://api.github.com/repos/Eshcar/hbase/issues{/number}","pulls_url":"https://api.github.com/repos/Eshcar/hbase/pulls{/number}","milestones_url":"https://api.github.com/repos/Eshcar/hbase/milestones{/number}","notifications_url":"https://api.github.com/repos/Eshcar/hbase/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/Eshcar/hbase/labels{/name}","releases_url":"https://api.github.com/repos/Eshcar/hbase/releases{/id}","deployments_url":"https://api.github.com/repos/Eshcar/hbase/deployments","created_at":"2015-02-10T08:50:25Z","updated_at":"2016-01-17T10:18:51Z","pushed_at":"2016-04-13T07:15:04Z","git_url":"git://github.com/Eshcar/hbase.git","ssh_url":"git@github.com:Eshcar/hbase.git","clone_url":"https://github.com/Eshcar/hbase.git","svn_url":"https://github.com/Eshcar/hbase","homepage":null,"size":217094,"stargazers_count":0,"watchers_count":0,"language":"Java","has_issues":false,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":1,"mirror_url":null,"open_issues_count":5,"forks":1,"open_issues":5,"watchers":0,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/Eshcar/hbase/pulls/6"},"html":{"href":"https://github.com/Eshcar/hbase/pull/6"},"issue":{"href":"https://api.github.com/repos/Eshcar/hbase/issues/6"},"comments":{"href":"https://api.github.com/repos/Eshcar/hbase/issues/6/comments"},"review_comments":{"href":"https://api.github.com/repos/Eshcar/hbase/pulls/6/comments"},"review_comment":{"href":"https://api.github.com/repos/Eshcar/hbase/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/Eshcar/hbase/pulls/6/commits"},"statuses":{"href":"https://api.github.com/repos/Eshcar/hbase/statuses/d2a9a20b70c96ecd20770aa9fff11c2b29b7b7da"}}}}