{"action":"closed","number":1259,"pull_request":{"url":"https://api.github.com/repos/ga4gh/server/pulls/1259","id":72942312,"html_url":"https://github.com/ga4gh/server/pull/1259","diff_url":"https://github.com/ga4gh/server/pull/1259.diff","patch_url":"https://github.com/ga4gh/server/pull/1259.patch","issue_url":"https://api.github.com/repos/ga4gh/server/issues/1259","number":1259,"state":"closed","locked":false,"title":"SqlAlchemy POC","user":{"login":"jeromekelleher","id":2664569,"avatar_url":"https://avatars.githubusercontent.com/u/2664569?v=3","gravatar_id":"","url":"https://api.github.com/users/jeromekelleher","html_url":"https://github.com/jeromekelleher","followers_url":"https://api.github.com/users/jeromekelleher/followers","following_url":"https://api.github.com/users/jeromekelleher/following{/other_user}","gists_url":"https://api.github.com/users/jeromekelleher/gists{/gist_id}","starred_url":"https://api.github.com/users/jeromekelleher/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jeromekelleher/subscriptions","organizations_url":"https://api.github.com/users/jeromekelleher/orgs","repos_url":"https://api.github.com/users/jeromekelleher/repos","events_url":"https://api.github.com/users/jeromekelleher/events{/privacy}","received_events_url":"https://api.github.com/users/jeromekelleher/received_events","type":"User","site_admin":false},"body":"This is a very early work in progress, and not intended for review. The PR is intended to show the direction that the SQLAlchemy refactor is going in, and hopefully get some early feedback from those that are interested.\r\n\r\nThe main idea is that we pull out our clunky object model for all of the objects in our registry, and use the SQLAlchemy library to do the heavy work for us. \r\n\r\n## What works\r\n\r\n- Rough outline model of Datasets, ReferenceSets and References.\r\n- Polymorphism within the DB for HtslibReferenceSets and SimulatedReferenceSets. We can store either in the DB, and we instantiate the correct object at run time when we need to access information for a specific information. This is the key requirement for more flexible storage backends.\r\n- Search and GET requests for references, referencesets and datasets. The paging algorithm has been reworked to use SQL natively, and is much cleaner.\r\n- Native support for multiple databases for the registry. I've tested this on Sqlite and Postgres and it works great on both. \r\n- Updated the repo manager CLI to use the new interface for the subset of data types supported. The repo manager now takes a DB URL in the form of a SQLAlchemy connection string. \r\n- Online updates to the DB. Updates to the registry from the CLI are now reflected in the server without needing a server restart.\r\n\r\n## TODO\r\n\r\n- A fair bit of refactoring and shuffling around of code.\r\n- Complete the rest of the data model (ReadGroupSets, VariantSets, etc).\r\n- Fix test cases.\r\n- Add some way of testing the postgres support using Travis.\r\n\r\n## Questions\r\n\r\n- Do we really need to worry about ID collisions between tables? I.e., does it actually matter if we have a  Dataset and a VariantSet with the same ID? I don't think we need uniqueness to fulfill the requirements of the server, and it adds a fair bit of complexity ensuring that they are unique.\r\n- How should we model things like ReferenceSet.source_accesssions? At the moment we embed some JSON into the table and don't worry about it. However, the right thing to do here would seem to be to model it with a table. It would surely be more forward-compatible to do this, if we ever need to get clever about accession numbers. \r\n- More generally, do we go for a full-on SQL model of all objects (like Experiment, VariantSetMetadata, etc), or do we continue using embedded JSON stored in a table? ","created_at":"2016-06-07T20:26:38Z","updated_at":"2016-09-26T13:33:25Z","closed_at":"2016-09-26T13:33:24Z","merged_at":null,"merge_commit_sha":"22754d076efc20048b3b9b69c8d71fb2c1841581","assignee":null,"assignees":[],"milestone":null,"commits_url":"https://api.github.com/repos/ga4gh/server/pulls/1259/commits","review_comments_url":"https://api.github.com/repos/ga4gh/server/pulls/1259/comments","review_comment_url":"https://api.github.com/repos/ga4gh/server/pulls/comments{/number}","comments_url":"https://api.github.com/repos/ga4gh/server/issues/1259/comments","statuses_url":"https://api.github.com/repos/ga4gh/server/statuses/383648b6ec2cf3f2eba43b99b342c3f59226d4d2","head":{"label":"jeromekelleher:sql-alchemy-poc","ref":"sql-alchemy-poc","sha":"383648b6ec2cf3f2eba43b99b342c3f59226d4d2","user":{"login":"jeromekelleher","id":2664569,"avatar_url":"https://avatars.githubusercontent.com/u/2664569?v=3","gravatar_id":"","url":"https://api.github.com/users/jeromekelleher","html_url":"https://github.com/jeromekelleher","followers_url":"https://api.github.com/users/jeromekelleher/followers","following_url":"https://api.github.com/users/jeromekelleher/following{/other_user}","gists_url":"https://api.github.com/users/jeromekelleher/gists{/gist_id}","starred_url":"https://api.github.com/users/jeromekelleher/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jeromekelleher/subscriptions","organizations_url":"https://api.github.com/users/jeromekelleher/orgs","repos_url":"https://api.github.com/users/jeromekelleher/repos","events_url":"https://api.github.com/users/jeromekelleher/events{/privacy}","received_events_url":"https://api.github.com/users/jeromekelleher/received_events","type":"User","site_admin":false},"repo":{"id":22882845,"name":"server","full_name":"jeromekelleher/server","owner":{"login":"jeromekelleher","id":2664569,"avatar_url":"https://avatars.githubusercontent.com/u/2664569?v=3","gravatar_id":"","url":"https://api.github.com/users/jeromekelleher","html_url":"https://github.com/jeromekelleher","followers_url":"https://api.github.com/users/jeromekelleher/followers","following_url":"https://api.github.com/users/jeromekelleher/following{/other_user}","gists_url":"https://api.github.com/users/jeromekelleher/gists{/gist_id}","starred_url":"https://api.github.com/users/jeromekelleher/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jeromekelleher/subscriptions","organizations_url":"https://api.github.com/users/jeromekelleher/orgs","repos_url":"https://api.github.com/users/jeromekelleher/repos","events_url":"https://api.github.com/users/jeromekelleher/events{/privacy}","received_events_url":"https://api.github.com/users/jeromekelleher/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/jeromekelleher/server","description":"A reference implementation of the APIs defined in the schemas repository.","fork":true,"url":"https://api.github.com/repos/jeromekelleher/server","forks_url":"https://api.github.com/repos/jeromekelleher/server/forks","keys_url":"https://api.github.com/repos/jeromekelleher/server/keys{/key_id}","collaborators_url":"https://api.github.com/repos/jeromekelleher/server/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/jeromekelleher/server/teams","hooks_url":"https://api.github.com/repos/jeromekelleher/server/hooks","issue_events_url":"https://api.github.com/repos/jeromekelleher/server/issues/events{/number}","events_url":"https://api.github.com/repos/jeromekelleher/server/events","assignees_url":"https://api.github.com/repos/jeromekelleher/server/assignees{/user}","branches_url":"https://api.github.com/repos/jeromekelleher/server/branches{/branch}","tags_url":"https://api.github.com/repos/jeromekelleher/server/tags","blobs_url":"https://api.github.com/repos/jeromekelleher/server/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/jeromekelleher/server/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/jeromekelleher/server/git/refs{/sha}","trees_url":"https://api.github.com/repos/jeromekelleher/server/git/trees{/sha}","statuses_url":"https://api.github.com/repos/jeromekelleher/server/statuses/{sha}","languages_url":"https://api.github.com/repos/jeromekelleher/server/languages","stargazers_url":"https://api.github.com/repos/jeromekelleher/server/stargazers","contributors_url":"https://api.github.com/repos/jeromekelleher/server/contributors","subscribers_url":"https://api.github.com/repos/jeromekelleher/server/subscribers","subscription_url":"https://api.github.com/repos/jeromekelleher/server/subscription","commits_url":"https://api.github.com/repos/jeromekelleher/server/commits{/sha}","git_commits_url":"https://api.github.com/repos/jeromekelleher/server/git/commits{/sha}","comments_url":"https://api.github.com/repos/jeromekelleher/server/comments{/number}","issue_comment_url":"https://api.github.com/repos/jeromekelleher/server/issues/comments{/number}","contents_url":"https://api.github.com/repos/jeromekelleher/server/contents/{+path}","compare_url":"https://api.github.com/repos/jeromekelleher/server/compare/{base}...{head}","merges_url":"https://api.github.com/repos/jeromekelleher/server/merges","archive_url":"https://api.github.com/repos/jeromekelleher/server/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/jeromekelleher/server/downloads","issues_url":"https://api.github.com/repos/jeromekelleher/server/issues{/number}","pulls_url":"https://api.github.com/repos/jeromekelleher/server/pulls{/number}","milestones_url":"https://api.github.com/repos/jeromekelleher/server/milestones{/number}","notifications_url":"https://api.github.com/repos/jeromekelleher/server/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/jeromekelleher/server/labels{/name}","releases_url":"https://api.github.com/repos/jeromekelleher/server/releases{/id}","deployments_url":"https://api.github.com/repos/jeromekelleher/server/deployments","created_at":"2014-08-12T15:39:03Z","updated_at":"2015-02-12T09:23:10Z","pushed_at":"2016-08-18T16:41:26Z","git_url":"git://github.com/jeromekelleher/server.git","ssh_url":"git@github.com:jeromekelleher/server.git","clone_url":"https://github.com/jeromekelleher/server.git","svn_url":"https://github.com/jeromekelleher/server","homepage":null,"size":9043,"stargazers_count":1,"watchers_count":1,"language":"Python","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":1,"mirror_url":null,"open_issues_count":0,"forks":1,"open_issues":0,"watchers":1,"default_branch":"master"}},"base":{"label":"ga4gh:master","ref":"master","sha":"579ab7a380039314eec93f3d8e4afcc0b379276a","user":{"login":"ga4gh","id":6921777,"avatar_url":"https://avatars.githubusercontent.com/u/6921777?v=3","gravatar_id":"","url":"https://api.github.com/users/ga4gh","html_url":"https://github.com/ga4gh","followers_url":"https://api.github.com/users/ga4gh/followers","following_url":"https://api.github.com/users/ga4gh/following{/other_user}","gists_url":"https://api.github.com/users/ga4gh/gists{/gist_id}","starred_url":"https://api.github.com/users/ga4gh/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ga4gh/subscriptions","organizations_url":"https://api.github.com/users/ga4gh/orgs","repos_url":"https://api.github.com/users/ga4gh/repos","events_url":"https://api.github.com/users/ga4gh/events{/privacy}","received_events_url":"https://api.github.com/users/ga4gh/received_events","type":"Organization","site_admin":false},"repo":{"id":22882621,"name":"server","full_name":"ga4gh/server","owner":{"login":"ga4gh","id":6921777,"avatar_url":"https://avatars.githubusercontent.com/u/6921777?v=3","gravatar_id":"","url":"https://api.github.com/users/ga4gh","html_url":"https://github.com/ga4gh","followers_url":"https://api.github.com/users/ga4gh/followers","following_url":"https://api.github.com/users/ga4gh/following{/other_user}","gists_url":"https://api.github.com/users/ga4gh/gists{/gist_id}","starred_url":"https://api.github.com/users/ga4gh/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ga4gh/subscriptions","organizations_url":"https://api.github.com/users/ga4gh/orgs","repos_url":"https://api.github.com/users/ga4gh/repos","events_url":"https://api.github.com/users/ga4gh/events{/privacy}","received_events_url":"https://api.github.com/users/ga4gh/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/ga4gh/server","description":"A reference implementation of the APIs defined in the schemas repository.","fork":false,"url":"https://api.github.com/repos/ga4gh/server","forks_url":"https://api.github.com/repos/ga4gh/server/forks","keys_url":"https://api.github.com/repos/ga4gh/server/keys{/key_id}","collaborators_url":"https://api.github.com/repos/ga4gh/server/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/ga4gh/server/teams","hooks_url":"https://api.github.com/repos/ga4gh/server/hooks","issue_events_url":"https://api.github.com/repos/ga4gh/server/issues/events{/number}","events_url":"https://api.github.com/repos/ga4gh/server/events","assignees_url":"https://api.github.com/repos/ga4gh/server/assignees{/user}","branches_url":"https://api.github.com/repos/ga4gh/server/branches{/branch}","tags_url":"https://api.github.com/repos/ga4gh/server/tags","blobs_url":"https://api.github.com/repos/ga4gh/server/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/ga4gh/server/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/ga4gh/server/git/refs{/sha}","trees_url":"https://api.github.com/repos/ga4gh/server/git/trees{/sha}","statuses_url":"https://api.github.com/repos/ga4gh/server/statuses/{sha}","languages_url":"https://api.github.com/repos/ga4gh/server/languages","stargazers_url":"https://api.github.com/repos/ga4gh/server/stargazers","contributors_url":"https://api.github.com/repos/ga4gh/server/contributors","subscribers_url":"https://api.github.com/repos/ga4gh/server/subscribers","subscription_url":"https://api.github.com/repos/ga4gh/server/subscription","commits_url":"https://api.github.com/repos/ga4gh/server/commits{/sha}","git_commits_url":"https://api.github.com/repos/ga4gh/server/git/commits{/sha}","comments_url":"https://api.github.com/repos/ga4gh/server/comments{/number}","issue_comment_url":"https://api.github.com/repos/ga4gh/server/issues/comments{/number}","contents_url":"https://api.github.com/repos/ga4gh/server/contents/{+path}","compare_url":"https://api.github.com/repos/ga4gh/server/compare/{base}...{head}","merges_url":"https://api.github.com/repos/ga4gh/server/merges","archive_url":"https://api.github.com/repos/ga4gh/server/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/ga4gh/server/downloads","issues_url":"https://api.github.com/repos/ga4gh/server/issues{/number}","pulls_url":"https://api.github.com/repos/ga4gh/server/pulls{/number}","milestones_url":"https://api.github.com/repos/ga4gh/server/milestones{/number}","notifications_url":"https://api.github.com/repos/ga4gh/server/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/ga4gh/server/labels{/name}","releases_url":"https://api.github.com/repos/ga4gh/server/releases{/id}","deployments_url":"https://api.github.com/repos/ga4gh/server/deployments","created_at":"2014-08-12T15:31:49Z","updated_at":"2016-09-23T18:25:29Z","pushed_at":"2016-09-22T20:47:36Z","git_url":"git://github.com/ga4gh/server.git","ssh_url":"git@github.com:ga4gh/server.git","clone_url":"https://github.com/ga4gh/server.git","svn_url":"https://github.com/ga4gh/server","homepage":null,"size":11420,"stargazers_count":73,"watchers_count":73,"language":"Python","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":93,"mirror_url":null,"open_issues_count":133,"forks":93,"open_issues":133,"watchers":73,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/ga4gh/server/pulls/1259"},"html":{"href":"https://github.com/ga4gh/server/pull/1259"},"issue":{"href":"https://api.github.com/repos/ga4gh/server/issues/1259"},"comments":{"href":"https://api.github.com/repos/ga4gh/server/issues/1259/comments"},"review_comments":{"href":"https://api.github.com/repos/ga4gh/server/pulls/1259/comments"},"review_comment":{"href":"https://api.github.com/repos/ga4gh/server/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/ga4gh/server/pulls/1259/commits"},"statuses":{"href":"https://api.github.com/repos/ga4gh/server/statuses/383648b6ec2cf3f2eba43b99b342c3f59226d4d2"}},"merged":false,"mergeable":false,"mergeable_state":"dirty","merged_by":null,"comments":26,"review_comments":9,"commits":1,"additions":5694,"deletions":8569,"changed_files":76}}
{"actor":{"display_login":"dcolligan"}}