{"action":"created","issue":{"url":"https://api.github.com/repos/couchbase/sync_gateway/issues/1648","repository_url":"https://api.github.com/repos/couchbase/sync_gateway","labels_url":"https://api.github.com/repos/couchbase/sync_gateway/issues/1648/labels{/name}","comments_url":"https://api.github.com/repos/couchbase/sync_gateway/issues/1648/comments","events_url":"https://api.github.com/repos/couchbase/sync_gateway/issues/1648/events","html_url":"https://github.com/couchbase/sync_gateway/issues/1648","id":139962109,"number":1648,"title":"Cleanup task for obsolete attachments","user":{"login":"ajres","id":7745210,"avatar_url":"https://avatars.githubusercontent.com/u/7745210?v=3","gravatar_id":"","url":"https://api.github.com/users/ajres","html_url":"https://github.com/ajres","followers_url":"https://api.github.com/users/ajres/followers","following_url":"https://api.github.com/users/ajres/following{/other_user}","gists_url":"https://api.github.com/users/ajres/gists{/gist_id}","starred_url":"https://api.github.com/users/ajres/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ajres/subscriptions","organizations_url":"https://api.github.com/users/ajres/orgs","repos_url":"https://api.github.com/users/ajres/repos","events_url":"https://api.github.com/users/ajres/events{/privacy}","received_events_url":"https://api.github.com/users/ajres/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/couchbase/sync_gateway/labels/enhancement","name":"enhancement","color":"84b6eb"},{"url":"https://api.github.com/repos/couchbase/sync_gateway/labels/in%20progress","name":"in progress","color":"ededed"},{"url":"https://api.github.com/repos/couchbase/sync_gateway/labels/P3:%20low","name":"P3: low","color":"bfd4f2"}],"state":"open","locked":false,"assignee":{"login":"adamcfraser","id":8866453,"avatar_url":"https://avatars.githubusercontent.com/u/8866453?v=3","gravatar_id":"","url":"https://api.github.com/users/adamcfraser","html_url":"https://github.com/adamcfraser","followers_url":"https://api.github.com/users/adamcfraser/followers","following_url":"https://api.github.com/users/adamcfraser/following{/other_user}","gists_url":"https://api.github.com/users/adamcfraser/gists{/gist_id}","starred_url":"https://api.github.com/users/adamcfraser/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/adamcfraser/subscriptions","organizations_url":"https://api.github.com/users/adamcfraser/orgs","repos_url":"https://api.github.com/users/adamcfraser/repos","events_url":"https://api.github.com/users/adamcfraser/events{/privacy}","received_events_url":"https://api.github.com/users/adamcfraser/received_events","type":"User","site_admin":false},"assignees":[{"login":"adamcfraser","id":8866453,"avatar_url":"https://avatars.githubusercontent.com/u/8866453?v=3","gravatar_id":"","url":"https://api.github.com/users/adamcfraser","html_url":"https://github.com/adamcfraser","followers_url":"https://api.github.com/users/adamcfraser/followers","following_url":"https://api.github.com/users/adamcfraser/following{/other_user}","gists_url":"https://api.github.com/users/adamcfraser/gists{/gist_id}","starred_url":"https://api.github.com/users/adamcfraser/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/adamcfraser/subscriptions","organizations_url":"https://api.github.com/users/adamcfraser/orgs","repos_url":"https://api.github.com/users/adamcfraser/repos","events_url":"https://api.github.com/users/adamcfraser/events{/privacy}","received_events_url":"https://api.github.com/users/adamcfraser/received_events","type":"User","site_admin":false}],"milestone":{"url":"https://api.github.com/repos/couchbase/sync_gateway/milestones/19","html_url":"https://github.com/couchbase/sync_gateway/milestones/1.4","labels_url":"https://api.github.com/repos/couchbase/sync_gateway/milestones/19/labels","id":1807674,"number":19,"title":"1.4","description":"","creator":{"login":"adamcfraser","id":8866453,"avatar_url":"https://avatars.githubusercontent.com/u/8866453?v=3","gravatar_id":"","url":"https://api.github.com/users/adamcfraser","html_url":"https://github.com/adamcfraser","followers_url":"https://api.github.com/users/adamcfraser/followers","following_url":"https://api.github.com/users/adamcfraser/following{/other_user}","gists_url":"https://api.github.com/users/adamcfraser/gists{/gist_id}","starred_url":"https://api.github.com/users/adamcfraser/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/adamcfraser/subscriptions","organizations_url":"https://api.github.com/users/adamcfraser/orgs","repos_url":"https://api.github.com/users/adamcfraser/repos","events_url":"https://api.github.com/users/adamcfraser/events{/privacy}","received_events_url":"https://api.github.com/users/adamcfraser/received_events","type":"User","site_admin":false},"open_issues":46,"closed_issues":1,"state":"open","created_at":"2016-06-03T18:12:47Z","updated_at":"2016-08-26T14:20:33Z","due_on":null,"closed_at":null},"comments":4,"created_at":"2016-03-10T17:44:25Z","updated_at":"2016-09-01T22:34:09Z","closed_at":null,"body":"When a document is deleted or an attachment removed the binary document remains in the underlying CBS bucket.\n\nFor apps that create and delete docs with attachments on a regular basis this can have a big impact on storage.\n\nThere is already a _vacuum DB endpoint which is meant to purge orphaned attachments but it is currently not implemented.\n\n```\n// Deletes all orphaned CouchDB attachments not used by any revisions.\nfunc VacuumAttachments(bucket base.Bucket) (int, error) {\n\treturn 0, base.HTTPErrorf(http.StatusNotImplemented, \"Vacuum is temporarily out of order\")\n}\n```"},"comment":{"url":"https://api.github.com/repos/couchbase/sync_gateway/issues/comments/244233132","html_url":"https://github.com/couchbase/sync_gateway/issues/1648#issuecomment-244233132","issue_url":"https://api.github.com/repos/couchbase/sync_gateway/issues/1648","id":244233132,"user":{"login":"adamcfraser","id":8866453,"avatar_url":"https://avatars.githubusercontent.com/u/8866453?v=3","gravatar_id":"","url":"https://api.github.com/users/adamcfraser","html_url":"https://github.com/adamcfraser","followers_url":"https://api.github.com/users/adamcfraser/followers","following_url":"https://api.github.com/users/adamcfraser/following{/other_user}","gists_url":"https://api.github.com/users/adamcfraser/gists{/gist_id}","starred_url":"https://api.github.com/users/adamcfraser/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/adamcfraser/subscriptions","organizations_url":"https://api.github.com/users/adamcfraser/orgs","repos_url":"https://api.github.com/users/adamcfraser/repos","events_url":"https://api.github.com/users/adamcfraser/events{/privacy}","received_events_url":"https://api.github.com/users/adamcfraser/received_events","type":"User","site_admin":false},"created_at":"2016-09-01T22:34:09Z","updated_at":"2016-09-01T22:34:09Z","body":"Original issue was filed as on https://github.com/couchbase/sync_gateway/issues/64.  The main design issue remaining is handling concurrency issues during GC processing, specifically:\r\n\r\n> Initial identification of orphaned attachments is relatively straightforward using a view, but there isn't a good solution for the associated concurrency problems when we try to perform the deletes based on the results of that view query:\r\n> \r\n>  1. A new document arrives referencing an attachment that's targeted for deletion while GC is in progress.\r\n>  2. An incoming new attachment gets incorrectly flagged for delete by the GC, since it's committed before the document is committed.\r\n> \r\n> In particular this is an issue for a multi-node SG cluster, where it's harder to do a 'stop the world' for attachments across the cluster while the GC is running.  We also need to ensure the solution only has a performance impact while GC is running, and has a negligible impact on standard attachment processing (i.e. we want to avoid significant work to lookup the attachment state during a normal write)"}}
{"actor":{"display_login":"adamcfraser"}}