{"action":"opened","number":59,"pull_request":{"url":"https://api.github.com/repos/icatproject/ids.server/pulls/59","id":92146316,"html_url":"https://github.com/icatproject/ids.server/pull/59","diff_url":"https://github.com/icatproject/ids.server/pull/59.diff","patch_url":"https://github.com/icatproject/ids.server/pull/59.patch","issue_url":"https://api.github.com/repos/icatproject/ids.server/issues/59","number":59,"state":"open","locked":false,"title":"[WIP] Implement the new locking mechanism","user":{"login":"RKrahl","id":11837376,"avatar_url":"https://avatars.githubusercontent.com/u/11837376?v=3","gravatar_id":"","url":"https://api.github.com/users/RKrahl","html_url":"https://github.com/RKrahl","followers_url":"https://api.github.com/users/RKrahl/followers","following_url":"https://api.github.com/users/RKrahl/following{/other_user}","gists_url":"https://api.github.com/users/RKrahl/gists{/gist_id}","starred_url":"https://api.github.com/users/RKrahl/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/RKrahl/subscriptions","organizations_url":"https://api.github.com/users/RKrahl/orgs","repos_url":"https://api.github.com/users/RKrahl/repos","events_url":"https://api.github.com/users/RKrahl/events{/privacy}","received_events_url":"https://api.github.com/users/RKrahl/received_events","type":"User","site_admin":false},"body":"**Note:** This is not yet ready, there is some tidying to be done before merging into master.  In particular, the `test` directory and the `NOTES.rst` file do not belong into the final sources and should be removed.  But otherwise it seem to be ready and I wanted to give you the opportunity to review the code and to test it before finalizing it, so I open this PR now.\r\n\r\nThis is the new locking mechanism as [discussed F2F in Copenhagen](https://icatproject.org/wp-content/uploads/2016/04/ids-locking.pdf).  In Copenhagen, I also made the proposal to add a plugin call so that the internal locking can be combined with file system locks to also synchronize with external processes.  This additional option could be coded on top of this PR later on.\r\n\r\nThere are some things to note:\r\n\r\n- `LockManager`: this is the central new class to manage the locks and to obtain the locks from.\r\n- Scope: Locks are always set on whole datasets, even in the case of storageUnit datafile.  This was also the case in the old implementation.  It would make things rather complicated to change this, so I suggest to leave it as is.\r\n- `AlreadyLockedException`: the `LockManager.lock()` method throws this exception if it can't obtain the lock because there is already another lock.  For the moment, the definition of the exception class is nested inside the `LockManager`.  There might be a better better place though.\r\n- `getServiceStatus`: This web service call used to return something like:\r\n  ```\r\n  {'opsQueue': [{'request': 'WRITE_THEN_ARCHIVE_REQUESTED', 'data': '911105/911235 (ESNF/12100409-ST/1.1-P/test_race_upload_archive)'}], 'lockCount': 1, 'lockedIds': [911235]}\r\n  ```\r\n  Note the field `lockedIds` which is a list of all datasets ids having a lock set.  The new implementation now returns:\r\n  ```\r\n  {'opsQueue': [{'request': 'ARCHIVE_REQUESTED', 'data': '911105/911238 (ESNF/12100409-ST/1.1-P/test_race_upload_archive)'}], 'lockCount': 1, 'locks': [{'id': 911238, 'count': 2, 'type': 'SHARED'}]}\r\n  ```\r\n  The field `lockedIds` has been renamed to `locks`.  The entries in this list now have the three attributes `id`, `type`, and `count`, where `id` is the dataset id, `type` is either `SHARED` or `EXCLUSIVE`, and `count` is the number of locks on this dataset.  I think this additional information might be useful.  But if compatibility with the old behavior is needed, it's essentially one line of code to change this.\r\n- Exceptions: Web service calls may now fail because the corresponding data is locked.  They will throw a `DataNotOnlineException` for the moment.  While the definition of the error condition might not quite match this case, from the client's point of view it's the same: it is a temporary failure and the client may retry the same request later on.  We may add another more specific error class for this purpose.\r\n  Note that the delete web service call used to throw `BadRequestException` in this case in the old implementation.  I'd say, this was just wrong.\r\n- Testing: I didn't add any junit tests.  First because I didn't learned the ropes of the test setup by now and can't run the test here either.  Secondly because it would be rather difficult to formulate suitable test cases in the first place.  In most situations, the effect of the locking can only be verified by checking the ids.log in Glassfish, but it is largely invisible on the client side.  I added a provisional `test` directory with some Python scripts, just to document things.  But this is certainly not the right place to put it and should be removed again before merging.","created_at":"2016-11-03T12:46:56Z","updated_at":"2016-11-03T12:46:56Z","closed_at":null,"merged_at":null,"merge_commit_sha":null,"assignee":null,"assignees":[],"milestone":null,"commits_url":"https://api.github.com/repos/icatproject/ids.server/pulls/59/commits","review_comments_url":"https://api.github.com/repos/icatproject/ids.server/pulls/59/comments","review_comment_url":"https://api.github.com/repos/icatproject/ids.server/pulls/comments{/number}","comments_url":"https://api.github.com/repos/icatproject/ids.server/issues/59/comments","statuses_url":"https://api.github.com/repos/icatproject/ids.server/statuses/962402178bbfb1bd550d892f7302b0b0d1326068","head":{"label":"RKrahl:locking","ref":"locking","sha":"962402178bbfb1bd550d892f7302b0b0d1326068","user":{"login":"RKrahl","id":11837376,"avatar_url":"https://avatars.githubusercontent.com/u/11837376?v=3","gravatar_id":"","url":"https://api.github.com/users/RKrahl","html_url":"https://github.com/RKrahl","followers_url":"https://api.github.com/users/RKrahl/followers","following_url":"https://api.github.com/users/RKrahl/following{/other_user}","gists_url":"https://api.github.com/users/RKrahl/gists{/gist_id}","starred_url":"https://api.github.com/users/RKrahl/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/RKrahl/subscriptions","organizations_url":"https://api.github.com/users/RKrahl/orgs","repos_url":"https://api.github.com/users/RKrahl/repos","events_url":"https://api.github.com/users/RKrahl/events{/privacy}","received_events_url":"https://api.github.com/users/RKrahl/received_events","type":"User","site_admin":false},"repo":{"id":55148677,"name":"ids.server","full_name":"RKrahl/ids.server","owner":{"login":"RKrahl","id":11837376,"avatar_url":"https://avatars.githubusercontent.com/u/11837376?v=3","gravatar_id":"","url":"https://api.github.com/users/RKrahl","html_url":"https://github.com/RKrahl","followers_url":"https://api.github.com/users/RKrahl/followers","following_url":"https://api.github.com/users/RKrahl/following{/other_user}","gists_url":"https://api.github.com/users/RKrahl/gists{/gist_id}","starred_url":"https://api.github.com/users/RKrahl/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/RKrahl/subscriptions","organizations_url":"https://api.github.com/users/RKrahl/orgs","repos_url":"https://api.github.com/users/RKrahl/repos","events_url":"https://api.github.com/users/RKrahl/events{/privacy}","received_events_url":"https://api.github.com/users/RKrahl/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/RKrahl/ids.server","description":"The server component of the ICAT Data Service","fork":true,"url":"https://api.github.com/repos/RKrahl/ids.server","forks_url":"https://api.github.com/repos/RKrahl/ids.server/forks","keys_url":"https://api.github.com/repos/RKrahl/ids.server/keys{/key_id}","collaborators_url":"https://api.github.com/repos/RKrahl/ids.server/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/RKrahl/ids.server/teams","hooks_url":"https://api.github.com/repos/RKrahl/ids.server/hooks","issue_events_url":"https://api.github.com/repos/RKrahl/ids.server/issues/events{/number}","events_url":"https://api.github.com/repos/RKrahl/ids.server/events","assignees_url":"https://api.github.com/repos/RKrahl/ids.server/assignees{/user}","branches_url":"https://api.github.com/repos/RKrahl/ids.server/branches{/branch}","tags_url":"https://api.github.com/repos/RKrahl/ids.server/tags","blobs_url":"https://api.github.com/repos/RKrahl/ids.server/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/RKrahl/ids.server/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/RKrahl/ids.server/git/refs{/sha}","trees_url":"https://api.github.com/repos/RKrahl/ids.server/git/trees{/sha}","statuses_url":"https://api.github.com/repos/RKrahl/ids.server/statuses/{sha}","languages_url":"https://api.github.com/repos/RKrahl/ids.server/languages","stargazers_url":"https://api.github.com/repos/RKrahl/ids.server/stargazers","contributors_url":"https://api.github.com/repos/RKrahl/ids.server/contributors","subscribers_url":"https://api.github.com/repos/RKrahl/ids.server/subscribers","subscription_url":"https://api.github.com/repos/RKrahl/ids.server/subscription","commits_url":"https://api.github.com/repos/RKrahl/ids.server/commits{/sha}","git_commits_url":"https://api.github.com/repos/RKrahl/ids.server/git/commits{/sha}","comments_url":"https://api.github.com/repos/RKrahl/ids.server/comments{/number}","issue_comment_url":"https://api.github.com/repos/RKrahl/ids.server/issues/comments{/number}","contents_url":"https://api.github.com/repos/RKrahl/ids.server/contents/{+path}","compare_url":"https://api.github.com/repos/RKrahl/ids.server/compare/{base}...{head}","merges_url":"https://api.github.com/repos/RKrahl/ids.server/merges","archive_url":"https://api.github.com/repos/RKrahl/ids.server/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/RKrahl/ids.server/downloads","issues_url":"https://api.github.com/repos/RKrahl/ids.server/issues{/number}","pulls_url":"https://api.github.com/repos/RKrahl/ids.server/pulls{/number}","milestones_url":"https://api.github.com/repos/RKrahl/ids.server/milestones{/number}","notifications_url":"https://api.github.com/repos/RKrahl/ids.server/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/RKrahl/ids.server/labels{/name}","releases_url":"https://api.github.com/repos/RKrahl/ids.server/releases{/id}","deployments_url":"https://api.github.com/repos/RKrahl/ids.server/deployments","created_at":"2016-03-31T12:26:31Z","updated_at":"2016-03-31T12:26:32Z","pushed_at":"2016-11-03T10:42:08Z","git_url":"git://github.com/RKrahl/ids.server.git","ssh_url":"git@github.com:RKrahl/ids.server.git","clone_url":"https://github.com/RKrahl/ids.server.git","svn_url":"https://github.com/RKrahl/ids.server","homepage":null,"size":1101,"stargazers_count":0,"watchers_count":0,"language":"Java","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"icatproject:master","ref":"master","sha":"910a284b675af01cae3e42161b64efa6b170f51a","user":{"login":"icatproject","id":11672419,"avatar_url":"https://avatars.githubusercontent.com/u/11672419?v=3","gravatar_id":"","url":"https://api.github.com/users/icatproject","html_url":"https://github.com/icatproject","followers_url":"https://api.github.com/users/icatproject/followers","following_url":"https://api.github.com/users/icatproject/following{/other_user}","gists_url":"https://api.github.com/users/icatproject/gists{/gist_id}","starred_url":"https://api.github.com/users/icatproject/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/icatproject/subscriptions","organizations_url":"https://api.github.com/users/icatproject/orgs","repos_url":"https://api.github.com/users/icatproject/repos","events_url":"https://api.github.com/users/icatproject/events{/privacy}","received_events_url":"https://api.github.com/users/icatproject/received_events","type":"Organization","site_admin":false},"repo":{"id":36434646,"name":"ids.server","full_name":"icatproject/ids.server","owner":{"login":"icatproject","id":11672419,"avatar_url":"https://avatars.githubusercontent.com/u/11672419?v=3","gravatar_id":"","url":"https://api.github.com/users/icatproject","html_url":"https://github.com/icatproject","followers_url":"https://api.github.com/users/icatproject/followers","following_url":"https://api.github.com/users/icatproject/following{/other_user}","gists_url":"https://api.github.com/users/icatproject/gists{/gist_id}","starred_url":"https://api.github.com/users/icatproject/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/icatproject/subscriptions","organizations_url":"https://api.github.com/users/icatproject/orgs","repos_url":"https://api.github.com/users/icatproject/repos","events_url":"https://api.github.com/users/icatproject/events{/privacy}","received_events_url":"https://api.github.com/users/icatproject/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/icatproject/ids.server","description":"The server component of the ICAT Data Service","fork":false,"url":"https://api.github.com/repos/icatproject/ids.server","forks_url":"https://api.github.com/repos/icatproject/ids.server/forks","keys_url":"https://api.github.com/repos/icatproject/ids.server/keys{/key_id}","collaborators_url":"https://api.github.com/repos/icatproject/ids.server/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/icatproject/ids.server/teams","hooks_url":"https://api.github.com/repos/icatproject/ids.server/hooks","issue_events_url":"https://api.github.com/repos/icatproject/ids.server/issues/events{/number}","events_url":"https://api.github.com/repos/icatproject/ids.server/events","assignees_url":"https://api.github.com/repos/icatproject/ids.server/assignees{/user}","branches_url":"https://api.github.com/repos/icatproject/ids.server/branches{/branch}","tags_url":"https://api.github.com/repos/icatproject/ids.server/tags","blobs_url":"https://api.github.com/repos/icatproject/ids.server/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/icatproject/ids.server/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/icatproject/ids.server/git/refs{/sha}","trees_url":"https://api.github.com/repos/icatproject/ids.server/git/trees{/sha}","statuses_url":"https://api.github.com/repos/icatproject/ids.server/statuses/{sha}","languages_url":"https://api.github.com/repos/icatproject/ids.server/languages","stargazers_url":"https://api.github.com/repos/icatproject/ids.server/stargazers","contributors_url":"https://api.github.com/repos/icatproject/ids.server/contributors","subscribers_url":"https://api.github.com/repos/icatproject/ids.server/subscribers","subscription_url":"https://api.github.com/repos/icatproject/ids.server/subscription","commits_url":"https://api.github.com/repos/icatproject/ids.server/commits{/sha}","git_commits_url":"https://api.github.com/repos/icatproject/ids.server/git/commits{/sha}","comments_url":"https://api.github.com/repos/icatproject/ids.server/comments{/number}","issue_comment_url":"https://api.github.com/repos/icatproject/ids.server/issues/comments{/number}","contents_url":"https://api.github.com/repos/icatproject/ids.server/contents/{+path}","compare_url":"https://api.github.com/repos/icatproject/ids.server/compare/{base}...{head}","merges_url":"https://api.github.com/repos/icatproject/ids.server/merges","archive_url":"https://api.github.com/repos/icatproject/ids.server/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/icatproject/ids.server/downloads","issues_url":"https://api.github.com/repos/icatproject/ids.server/issues{/number}","pulls_url":"https://api.github.com/repos/icatproject/ids.server/pulls{/number}","milestones_url":"https://api.github.com/repos/icatproject/ids.server/milestones{/number}","notifications_url":"https://api.github.com/repos/icatproject/ids.server/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/icatproject/ids.server/labels{/name}","releases_url":"https://api.github.com/repos/icatproject/ids.server/releases{/id}","deployments_url":"https://api.github.com/repos/icatproject/ids.server/deployments","created_at":"2015-05-28T11:43:13Z","updated_at":"2016-02-25T14:47:49Z","pushed_at":"2016-11-03T12:46:56Z","git_url":"git://github.com/icatproject/ids.server.git","ssh_url":"git@github.com:icatproject/ids.server.git","clone_url":"https://github.com/icatproject/ids.server.git","svn_url":"https://github.com/icatproject/ids.server","homepage":null,"size":725,"stargazers_count":0,"watchers_count":0,"language":"Java","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":1,"mirror_url":null,"open_issues_count":5,"forks":1,"open_issues":5,"watchers":0,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/icatproject/ids.server/pulls/59"},"html":{"href":"https://github.com/icatproject/ids.server/pull/59"},"issue":{"href":"https://api.github.com/repos/icatproject/ids.server/issues/59"},"comments":{"href":"https://api.github.com/repos/icatproject/ids.server/issues/59/comments"},"review_comments":{"href":"https://api.github.com/repos/icatproject/ids.server/pulls/59/comments"},"review_comment":{"href":"https://api.github.com/repos/icatproject/ids.server/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/icatproject/ids.server/pulls/59/commits"},"statuses":{"href":"https://api.github.com/repos/icatproject/ids.server/statuses/962402178bbfb1bd550d892f7302b0b0d1326068"}},"merged":false,"mergeable":null,"mergeable_state":"unknown","merged_by":null,"comments":0,"review_comments":0,"commits":28,"additions":21935,"deletions":367,"changed_files":22}}
{"actor":{"display_login":"RKrahl"}}