{"action":"created","issue":{"url":"https://api.github.com/repos/nextstrain/zika/issues/8","repository_url":"https://api.github.com/repos/nextstrain/zika","labels_url":"https://api.github.com/repos/nextstrain/zika/issues/8/labels{/name}","comments_url":"https://api.github.com/repos/nextstrain/zika/issues/8/comments","events_url":"https://api.github.com/repos/nextstrain/zika/issues/8/events","html_url":"https://github.com/nextstrain/zika/pull/8","id":339104168,"node_id":"MDExOlB1bGxSZXF1ZXN0MTk5ODcxOTgz","number":8,"title":"Streamline snakefile","user":{"login":"trvrb","id":1176109,"node_id":"MDQ6VXNlcjExNzYxMDk=","avatar_url":"https://avatars2.githubusercontent.com/u/1176109?v=4","gravatar_id":"","url":"https://api.github.com/users/trvrb","html_url":"https://github.com/trvrb","followers_url":"https://api.github.com/users/trvrb/followers","following_url":"https://api.github.com/users/trvrb/following{/other_user}","gists_url":"https://api.github.com/users/trvrb/gists{/gist_id}","starred_url":"https://api.github.com/users/trvrb/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/trvrb/subscriptions","organizations_url":"https://api.github.com/users/trvrb/orgs","repos_url":"https://api.github.com/users/trvrb/repos","events_url":"https://api.github.com/users/trvrb/events{/privacy}","received_events_url":"https://api.github.com/users/trvrb/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-07-07T00:48:36Z","updated_at":"2018-07-09T15:07:40Z","closed_at":null,"author_association":"MEMBER","pull_request":{"url":"https://api.github.com/repos/nextstrain/zika/pulls/8","html_url":"https://github.com/nextstrain/zika/pull/8","diff_url":"https://github.com/nextstrain/zika/pull/8.diff","patch_url":"https://github.com/nextstrain/zika/pull/8.patch"},"body":"Hi all. This is my attempt at a best practices snakefile. It differs somewhat from @tsibley's version of best practices. The base here is almost exactly `augur/builds/zika/Snakefile`. Relative to @tsibley's version, I've made the following changes:\r\n\r\n#### 1. Remove use `field_map` to get `fasta_fields`\r\n\r\nEven if this is inherited as a class, I think it's way overly complicated. Much simpler to just have a `fasta_fields` param in `download` and a `fasta_fields` param in `parse`. This makes it obvious what's going on and removes distracting boilerplate. This seemed too bespoke for a single use case of fauna vs augur parse as well.\r\n\r\n#### 2. Remove `config.yaml` and move params to snakefile\r\n\r\nI understand that `config.yaml` is commonly used for snakemake. And I did work with this a bunch, but I just couldn't see what we really gained from it. Whenever a rule needs updating it then adds redirection in going between the snakefile and `config.yaml`. I think better to have the parameters _right there_ in the rule. A single file with simple string parameters like `group_by = \"country year month\"` is less intimidating than two files and snakemake parameters `group_by = config['filter']['group_by']`.\r\n\r\nI understand that having `config.yaml` allows overwriting with `config_local.yaml`, but I actually couldn't see this run that often. And the command line override of `snakemake --config filter.sequences_per_group=1` seemed too verbose to be really useful.\r\n\r\n#### 3. Remove `credentials` from config\r\n\r\nWith this PR, the build now works by setting environment variables `RETHINK_HOST` and `RETHINK_AUTH_KEY`. These are referenced by fauna when running. The snakefile / config doesn't need to include them.\r\n\r\n#### 4. Remove `:p` from shell script strings\r\n\r\nI understand that `:p` is necessary to protect in the case of whitespace in file names. However, I am very comfortable with requiring that file names not include spaces.\r\n\r\n#### 5. Flatten `data/`, `results/` and `auspice/`\r\n\r\nPer issue #5. \r\n\r\n-------------------------------\r\n\r\nI really wanted the nextstrain.org snakemake files to be super interpretable and a showcase for how builds are run.\r\n\r\nI don't think all PRs need this much sign off, but it would be great if we could agree on this. Once we are in agreement, we can start copying out `nextstrain/tb`, `nextstrain/ebola`, etc...\r\n\r\n_Travis CI is failing due to out-of-date Docker image._"},"comment":{"url":"https://api.github.com/repos/nextstrain/zika/issues/comments/403512062","html_url":"https://github.com/nextstrain/zika/pull/8#issuecomment-403512062","issue_url":"https://api.github.com/repos/nextstrain/zika/issues/8","id":403512062,"node_id":"MDEyOklzc3VlQ29tbWVudDQwMzUxMjA2Mg==","user":{"login":"jameshadfield","id":8350992,"node_id":"MDQ6VXNlcjgzNTA5OTI=","avatar_url":"https://avatars1.githubusercontent.com/u/8350992?v=4","gravatar_id":"","url":"https://api.github.com/users/jameshadfield","html_url":"https://github.com/jameshadfield","followers_url":"https://api.github.com/users/jameshadfield/followers","following_url":"https://api.github.com/users/jameshadfield/following{/other_user}","gists_url":"https://api.github.com/users/jameshadfield/gists{/gist_id}","starred_url":"https://api.github.com/users/jameshadfield/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jameshadfield/subscriptions","organizations_url":"https://api.github.com/users/jameshadfield/orgs","repos_url":"https://api.github.com/users/jameshadfield/repos","events_url":"https://api.github.com/users/jameshadfield/events{/privacy}","received_events_url":"https://api.github.com/users/jameshadfield/received_events","type":"User","site_admin":false},"created_at":"2018-07-09T15:07:40Z","updated_at":"2018-07-09T15:07:40Z","author_association":"MEMBER","body":"See previous discussion on this topic [here](https://gist.github.com/tsibley/0e760b9f7d7955fc9d48ad87728a8af4)\r\n\r\nI really like the informative messages printed by nextstrain/zika's snakefile (e.g. https://github.com/nextstrain/zika/blob/master/Snakefile#L169). Can we bring them back?\r\n\r\nI agree with 1 & 5, my comment regarding 2 is [here](https://gist.github.com/tsibley/0e760b9f7d7955fc9d48ad87728a8af4#gistcomment-2633823), on point 4 i'm ambivalent. \r\n\r\nAt the end of the day, this is our suggested template & how the tutorial will be written. People are free to construct their own snakemake structures, use another build system, or just run augur commands in the terminal ;)"}}
{"actor":{"display_login":"jameshadfield"}}