{"action":"created","issue":{"url":"https://api.github.com/repos/galaxyproject/galaxy/issues/5669","repository_url":"https://api.github.com/repos/galaxyproject/galaxy","labels_url":"https://api.github.com/repos/galaxyproject/galaxy/issues/5669/labels{/name}","comments_url":"https://api.github.com/repos/galaxyproject/galaxy/issues/5669/comments","events_url":"https://api.github.com/repos/galaxyproject/galaxy/issues/5669/events","html_url":"https://github.com/galaxyproject/galaxy/pull/5669","id":303634468,"number":5669,"title":"Add script for propagating hashtags in a Galaxy history","user":{"login":"anuprulez","id":3022518,"avatar_url":"https://avatars2.githubusercontent.com/u/3022518?v=4","gravatar_id":"","url":"https://api.github.com/users/anuprulez","html_url":"https://github.com/anuprulez","followers_url":"https://api.github.com/users/anuprulez/followers","following_url":"https://api.github.com/users/anuprulez/following{/other_user}","gists_url":"https://api.github.com/users/anuprulez/gists{/gist_id}","starred_url":"https://api.github.com/users/anuprulez/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/anuprulez/subscriptions","organizations_url":"https://api.github.com/users/anuprulez/orgs","repos_url":"https://api.github.com/users/anuprulez/repos","events_url":"https://api.github.com/users/anuprulez/events{/privacy}","received_events_url":"https://api.github.com/users/anuprulez/received_events","type":"User","site_admin":false},"labels":[{"id":276600485,"url":"https://api.github.com/repos/galaxyproject/galaxy/labels/triage","name":"triage","color":"fbca04","default":false}],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":{"url":"https://api.github.com/repos/galaxyproject/galaxy/milestones/10","html_url":"https://github.com/galaxyproject/galaxy/milestone/10","labels_url":"https://api.github.com/repos/galaxyproject/galaxy/milestones/10/labels","id":2883981,"number":10,"title":"18.05","description":"","creator":{"login":"martenson","id":1814954,"avatar_url":"https://avatars2.githubusercontent.com/u/1814954?v=4","gravatar_id":"","url":"https://api.github.com/users/martenson","html_url":"https://github.com/martenson","followers_url":"https://api.github.com/users/martenson/followers","following_url":"https://api.github.com/users/martenson/following{/other_user}","gists_url":"https://api.github.com/users/martenson/gists{/gist_id}","starred_url":"https://api.github.com/users/martenson/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/martenson/subscriptions","organizations_url":"https://api.github.com/users/martenson/orgs","repos_url":"https://api.github.com/users/martenson/repos","events_url":"https://api.github.com/users/martenson/events{/privacy}","received_events_url":"https://api.github.com/users/martenson/received_events","type":"User","site_admin":false},"open_issues":55,"closed_issues":113,"state":"open","created_at":"2017-11-02T15:59:46Z","updated_at":"2018-03-20T19:43:29Z","due_on":"2018-05-01T07:00:00Z","closed_at":null},"comments":5,"created_at":"2018-03-08T21:16:16Z","updated_at":"2018-03-20T19:45:08Z","closed_at":"2018-03-20T19:43:29Z","author_association":"MEMBER","pull_request":{"url":"https://api.github.com/repos/galaxyproject/galaxy/pulls/5669","html_url":"https://github.com/galaxyproject/galaxy/pull/5669","diff_url":"https://github.com/galaxyproject/galaxy/pull/5669.diff","patch_url":"https://github.com/galaxyproject/galaxy/pull/5669.patch"},"body":"## Background:\r\n\r\nTo process data using Galaxy, an raw dataset (HDA) is uploaded and few hashtags are added to make it better searchable and identifiable. Then, using different tools/workflows, this raw/input dataset is transformed into multiple new datasets. What if, after running dozens of tools, generating hundreds or thousands of datasets, you realise that you’ve forgotten to tag your input datasets? In this case, the new hashtags will not be propagated to the generated datasets automatically. We require an easy and automatic solution to assign correct hashtags to all of the downstream datasets generated from our inputs, whenever new hashtags are added to any of the datasets in our history. Before going into the further explanation, let’s assign the following labels to the types/names of datasets we would use:\r\n\r\n- Input datasets as parent datasets\r\n- Derived/generated datasets as child datasets\r\n\r\nThis script will solve the situations mentioned above, adding any new hashtags to all derived datasets. The script is written considering a Galaxy history as a tree (data structure) with input datasets as its roots and all the generated ones either as nodes or leaves of this tree. An example history ‘tree’ is shown in the figure below.\r\n<p align=\"center\">\r\n  <img src=\"https://user-images.githubusercontent.com/3022518/37175617-5382eec8-231a-11e8-98b2-d5988dce3bb4.png\">\r\n</p>                                                                \r\nIn the above figure, we can see one input dataset with multiple datasets created from this input, using different tools (Galaxy tools). There can be any number of levels (tree hierarchy) in the ‘history’ tree from the top to bottom.\r\n\r\n## How to use the script:\r\nIn order to use the script to propagate hashtags from parents to children, a user should remember the hierarchy of their datasets. Wherever hashtags are added to datasets, they will be propagated to all the children of that dataset.  These new hashtags will be appended to the already existing hashtags for each child dataset. To use this script:\r\n\r\n1. Acquire your API key from your Galaxy.\r\n2. (Optionally) Get the ID of the history to apply hashtags to. If this id is not supplied, the script will use the most recently updated history.\r\n3. Install [BioBlend](https://bioblend.readthedocs.io/en/latest/#installation) on your machine.\r\n4. Execute: \r\n`python apply_tags.py <galaxy_url> <galaxy_api_key> <history_id optional>`\r\n\r\nHere, <galaxy_url> is the URL of your Galaxy instance. <galaxy_api_key> is the Galaxy API key you obtained from your user preferences. <history_id> is the encoded id of a Galaxy history. It is an optional parameter.\r\n\r\n## Example commands: \r\nPropagate hashtags in a specific history\r\n`python apply_tags.py \"https://usegalaxy.org\" \"*******************\" \"f2db41e1fa331b3e\"`\r\n\r\nPropagate hashtags in your most recently updated history\r\n`python apply_tags.py \"https://usegalaxy.org\" \"*******************\"`\r\n\r\n## A visual example:\r\n1. Upload a raw dataset using an upload tool and add a couple of hashtags (figure below the next point). Each arrow denotes the use of a Galaxy tool to transform one dataset into another.\r\n\r\n2. Run a few tools (say ToolA and ToolB) using the input dataset and create new (child) datasets (figure below).\r\n<p align=\"center\">\r\n  <img src=\"https://user-images.githubusercontent.com/3022518/37175698-99004f90-231a-11e8-8e4d-79109518cd10.png\">\r\n</p>\r\n\r\n3. Now, add new hashtags to the input dataset. The figure below shows new hashtags (#new_root_tag1, #new_root_tag2) getting added to the input dataset.\r\n<p align=\"center\">\r\n  <img src=\"https://user-images.githubusercontent.com/3022518/37175716-a531b600-231a-11e8-8088-a41cf46404f9.png\">\r\n</p>\r\n\r\n4. To propagate the new hashtags from parent to child datasets, execute the script using the example commands (here one should use the id of the history created above containing 3 datasets). We can see the child datasets now contain the new hashtags from their parent (figure below).\r\n<p align=\"center\">\r\n  <img src=\"https://user-images.githubusercontent.com/3022518/37175722-ab5794e6-231a-11e8-976b-2598eaadd9ce.png\">\r\n</p>\r\n\r\n## More complex example:\r\nIn the previous example, we saw a tree with one level of hierarchy. In the following visual, we would see a more complex example of inheriting hashtags from parents involving multiple levels in the history tree.(figure below).\r\n<p align=\"center\">\r\n  <img src=\"https://user-images.githubusercontent.com/3022518/37175736-b52df492-231a-11e8-9524-542208135533.png\">\r\n</p>\r\n\r\n## Propagate hashtags in Galaxy:\r\n<p align=\"center\">\r\n  <img src=\"https://user-images.githubusercontent.com/3022518/37175741-bb172a40-231a-11e8-835c-50166b6f11e3.png\">\r\n</p>   \r\n                                                                     \r\n1.fasta &rarr; 3:Replace Text on data 1 &rarr; 4: Text transformation on data 3\r\n\r\nIn the above figure, “1.fasta” is the root (input dataset). “3:Replace Text on data 1” is the child/derived dataset from the root. “4: Text transformation on data 3” is the child of the second dataset. The hashtags that we see are after executing the script on this history.\r\n\r\nThanks @torhou, @erasche and @bgruening \r\n\r\n**Note**: We are currently working to integrate this idea tightly into Galaxy as a UI feature.\r\n"},"comment":{"url":"https://api.github.com/repos/galaxyproject/galaxy/issues/comments/374731914","html_url":"https://github.com/galaxyproject/galaxy/pull/5669#issuecomment-374731914","issue_url":"https://api.github.com/repos/galaxyproject/galaxy/issues/5669","id":374731914,"user":{"login":"jmchilton","id":216771,"avatar_url":"https://avatars0.githubusercontent.com/u/216771?v=4","gravatar_id":"","url":"https://api.github.com/users/jmchilton","html_url":"https://github.com/jmchilton","followers_url":"https://api.github.com/users/jmchilton/followers","following_url":"https://api.github.com/users/jmchilton/following{/other_user}","gists_url":"https://api.github.com/users/jmchilton/gists{/gist_id}","starred_url":"https://api.github.com/users/jmchilton/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jmchilton/subscriptions","organizations_url":"https://api.github.com/users/jmchilton/orgs","repos_url":"https://api.github.com/users/jmchilton/repos","events_url":"https://api.github.com/users/jmchilton/events{/privacy}","received_events_url":"https://api.github.com/users/jmchilton/received_events","type":"User","site_admin":false},"created_at":"2018-03-20T19:45:08Z","updated_at":"2018-03-20T19:45:08Z","author_association":"OWNER","body":"Great - thanks for the neat script @anuprulez! I'm intrigued by the possibility to adding this to the UI - keep up the awesome work."}}
{"actor":{"display_login":"jmchilton"}}