{"action":"opened","issue":{"url":"https://api.github.com/repos/pmem/issues/issues/381","repository_url":"https://api.github.com/repos/pmem/issues","labels_url":"https://api.github.com/repos/pmem/issues/issues/381/labels{/name}","comments_url":"https://api.github.com/repos/pmem/issues/issues/381/comments","events_url":"https://api.github.com/repos/pmem/issues/issues/381/events","html_url":"https://github.com/pmem/issues/issues/381","id":197348259,"number":381,"title":"FEAT: async post-commit","user":{"login":"pbalcer","id":8775610,"avatar_url":"https://avatars.githubusercontent.com/u/8775610?v=3","gravatar_id":"","url":"https://api.github.com/users/pbalcer","html_url":"https://github.com/pbalcer","followers_url":"https://api.github.com/users/pbalcer/followers","following_url":"https://api.github.com/users/pbalcer/following{/other_user}","gists_url":"https://api.github.com/users/pbalcer/gists{/gist_id}","starred_url":"https://api.github.com/users/pbalcer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pbalcer/subscriptions","organizations_url":"https://api.github.com/users/pbalcer/orgs","repos_url":"https://api.github.com/users/pbalcer/repos","events_url":"https://api.github.com/users/pbalcer/events{/privacy}","received_events_url":"https://api.github.com/users/pbalcer/received_events","type":"User","site_admin":false},"labels":[{"id":121756459,"url":"https://api.github.com/repos/pmem/issues/labels/Type:%20Feature","name":"Type: Feature","color":"fc2929","default":false}],"state":"open","locked":false,"assignee":{"login":"pbalcer","id":8775610,"avatar_url":"https://avatars.githubusercontent.com/u/8775610?v=3","gravatar_id":"","url":"https://api.github.com/users/pbalcer","html_url":"https://github.com/pbalcer","followers_url":"https://api.github.com/users/pbalcer/followers","following_url":"https://api.github.com/users/pbalcer/following{/other_user}","gists_url":"https://api.github.com/users/pbalcer/gists{/gist_id}","starred_url":"https://api.github.com/users/pbalcer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pbalcer/subscriptions","organizations_url":"https://api.github.com/users/pbalcer/orgs","repos_url":"https://api.github.com/users/pbalcer/repos","events_url":"https://api.github.com/users/pbalcer/events{/privacy}","received_events_url":"https://api.github.com/users/pbalcer/received_events","type":"User","site_admin":false},"assignees":[{"login":"pbalcer","id":8775610,"avatar_url":"https://avatars.githubusercontent.com/u/8775610?v=3","gravatar_id":"","url":"https://api.github.com/users/pbalcer","html_url":"https://github.com/pbalcer","followers_url":"https://api.github.com/users/pbalcer/followers","following_url":"https://api.github.com/users/pbalcer/following{/other_user}","gists_url":"https://api.github.com/users/pbalcer/gists{/gist_id}","starred_url":"https://api.github.com/users/pbalcer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pbalcer/subscriptions","organizations_url":"https://api.github.com/users/pbalcer/orgs","repos_url":"https://api.github.com/users/pbalcer/repos","events_url":"https://api.github.com/users/pbalcer/events{/privacy}","received_events_url":"https://api.github.com/users/pbalcer/received_events","type":"User","site_admin":false}],"milestone":{"url":"https://api.github.com/repos/pmem/issues/milestones/6","html_url":"https://github.com/pmem/issues/milestone/6","labels_url":"https://api.github.com/repos/pmem/issues/milestones/6/labels","id":2212098,"number":6,"title":"1.4","description":"","creator":{"login":"krzycz","id":8184299,"avatar_url":"https://avatars.githubusercontent.com/u/8184299?v=3","gravatar_id":"","url":"https://api.github.com/users/krzycz","html_url":"https://github.com/krzycz","followers_url":"https://api.github.com/users/krzycz/followers","following_url":"https://api.github.com/users/krzycz/following{/other_user}","gists_url":"https://api.github.com/users/krzycz/gists{/gist_id}","starred_url":"https://api.github.com/users/krzycz/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/krzycz/subscriptions","organizations_url":"https://api.github.com/users/krzycz/orgs","repos_url":"https://api.github.com/users/krzycz/repos","events_url":"https://api.github.com/users/krzycz/events{/privacy}","received_events_url":"https://api.github.com/users/krzycz/received_events","type":"User","site_admin":false},"open_issues":4,"closed_issues":0,"state":"open","created_at":"2016-12-22T14:16:12Z","updated_at":"2016-12-23T10:44:18Z","due_on":"2017-06-30T07:00:00Z","closed_at":null},"comments":0,"created_at":"2016-12-23T10:44:18Z","updated_at":"2016-12-23T10:44:18Z","closed_at":null,"body":"# Asynchronous post commit\r\n\r\n## Rationale\r\nThe post-commit phase of the transaction, the purpose of which is to effectively clean the undo logs that were being used during the transaction, can take a lot of time, especially when the transaction has performed many TX_FREE operations, as the bulk of that operation is performed once the transaction finishes. The thing is though, the time-frame in which this phase is finished is completely irrelevant to the correctness of the system. Currently it is performed sequentially after finishing the transaction, performing the pre-commit phase (flushing of data), and marking the transaction as committed (on the on-media layout).\r\n\r\n## Description\r\nThe idea is to run the post-commit phase in a separate worker thread that runs in the background. The worker would perform everything that is currently performed in the post-commit phase of the transaction (cleanup the vectors, perform TX_FREE and so on) and additionally would release the lane once finished. \r\n\r\nThis mechanism would be opt-in and the user would have to provide the library with a thread that will be allowed to run for the entire life of the application.\r\n\r\nThis optimization makes the transactional free operation almost free from the perspective of the calling thread, as the actual free would be offloaded to a completely separate thread.\r\n\r\n## API Changes\r\nTwo new CTL entry point would be added:\r\n\"tx.post_commit.thread\", \"\"tx.post_commit.queue_depth\"\r\n\r\nThe thread entry points takes a pointer to an existing thread that runs the worker function and queue depths defines how many transactions can wait in the workers queue before every other calling thread would have to wait - this is to limit the potential issue with using too many lanes (as lanes would only be released once the worker thread finishes the task).\r\n\r\n## Implementation details\r\n\r\nThe worker thread would be implemented using a multiple produced single consumer queue (a circular buffer with queue depth defining its size). Due to the fact that the actual performance of the worker doesn't matter that much, it would spend most of its time sleeping and only periodically waiting to check if there's work to be performed.\r\n"}}
{"actor":{"display_login":"pbalcer"}}