{"action":"created","issue":{"url":"https://api.github.com/repos/msmbuilder/msmbuilder/issues/768","repository_url":"https://api.github.com/repos/msmbuilder/msmbuilder","labels_url":"https://api.github.com/repos/msmbuilder/msmbuilder/issues/768/labels{/name}","comments_url":"https://api.github.com/repos/msmbuilder/msmbuilder/issues/768/comments","events_url":"https://api.github.com/repos/msmbuilder/msmbuilder/issues/768/events","html_url":"https://github.com/msmbuilder/msmbuilder/pull/768","id":142113373,"number":768,"title":"Project Templates [WIP]","user":{"login":"mpharrigan","id":4967059,"avatar_url":"https://avatars.githubusercontent.com/u/4967059?v=3","gravatar_id":"","url":"https://api.github.com/users/mpharrigan","html_url":"https://github.com/mpharrigan","followers_url":"https://api.github.com/users/mpharrigan/followers","following_url":"https://api.github.com/users/mpharrigan/following{/other_user}","gists_url":"https://api.github.com/users/mpharrigan/gists{/gist_id}","starred_url":"https://api.github.com/users/mpharrigan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mpharrigan/subscriptions","organizations_url":"https://api.github.com/users/mpharrigan/orgs","repos_url":"https://api.github.com/users/mpharrigan/repos","events_url":"https://api.github.com/users/mpharrigan/events{/privacy}","received_events_url":"https://api.github.com/users/mpharrigan/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":{"url":"https://api.github.com/repos/msmbuilder/msmbuilder/milestones/6","html_url":"https://github.com/msmbuilder/msmbuilder/milestones/v3.5","labels_url":"https://api.github.com/repos/msmbuilder/msmbuilder/milestones/6/labels","id":1382452,"number":6,"title":"v3.5","description":"","creator":{"login":"mpharrigan","id":4967059,"avatar_url":"https://avatars.githubusercontent.com/u/4967059?v=3","gravatar_id":"","url":"https://api.github.com/users/mpharrigan","html_url":"https://github.com/mpharrigan","followers_url":"https://api.github.com/users/mpharrigan/followers","following_url":"https://api.github.com/users/mpharrigan/following{/other_user}","gists_url":"https://api.github.com/users/mpharrigan/gists{/gist_id}","starred_url":"https://api.github.com/users/mpharrigan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mpharrigan/subscriptions","organizations_url":"https://api.github.com/users/mpharrigan/orgs","repos_url":"https://api.github.com/users/mpharrigan/repos","events_url":"https://api.github.com/users/mpharrigan/events{/privacy}","received_events_url":"https://api.github.com/users/mpharrigan/received_events","type":"User","site_admin":false},"open_issues":20,"closed_issues":6,"state":"open","created_at":"2015-10-28T23:21:13Z","updated_at":"2016-04-06T18:33:54Z","due_on":"2016-05-30T07:00:00Z","closed_at":null},"comments":7,"created_at":"2016-03-20T00:06:46Z","updated_at":"2016-04-07T01:03:50Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/msmbuilder/msmbuilder/pulls/768","html_url":"https://github.com/msmbuilder/msmbuilder/pull/768","diff_url":"https://github.com/msmbuilder/msmbuilder/pull/768.diff","patch_url":"https://github.com/msmbuilder/msmbuilder/pull/768.patch"},"body":"This is a serious wip, but it's solid work towards making msmbuilder actually usable.\r\n\r\nThe general idea is we will develop a set of short scripts that do common tasks, the user will run `msmb SetUpProject`, and it'll dump a variety of scripts that will serve as a good starting point.\r\n\r\nThis PR also includes a new dataset idiom including a persistent pandas dataframe that keeps a mapping from a consistent set of keys (the dataframe index) to trajectory filenames and other metadata (topology filename, number of frames, whatever you want ... the user is encouraged to modify the \"generate-metadata\" script)\r\n\r\nData is still saved as a bunch of numpy files in a directory, but now instead of trying to walk the filesystem to find files, we use the dataframe to find everything we're looking for.\r\n\r\nMy grand plan is to get these scripts pretty solid for a \"generic\" project where trajectories are keyed on integers, and then start including fah-specific scripts [primarily modifying the metadata part] that set up a multi-indexed dataframe (proj,run,clone) and saves trajectory files in a proj/run/clone.npy hierarchy on the filesystem.\r\n\r\nThis PR ropes in #747 with modifications.\r\n\r\nI think the existing dataset infrastructure is overkill in terms of lazy-loading. The amount of GB of data is low for everything besides featurization, so we can shove our whole dataset into memory.\r\n\r\nFor example, 1 millisecond of data saved every 1 ns with 1000 features as 32-bit floats is only 4GB. That being said, we'll still have to do the initial featurization in a memory-efficient manner\r\n\r\nOne of my grander plans is to change from treating \"lists of timeseries-arrays\" as the thing that flows through our estimators to \"dictionaries of timeseries-arrays\". I imagine modifying our `transform` wrapper to return a list if a list is given and a dictionary if a dictionary is given, for backwards compatibility.\r\n\r\n1. the trajectories are inherently not in any order, as implied by a list\r\n2. meaningful keys are often non-sequential. This puts undue burden on the user to keep track of a mapping from \"real keys\" to list indices\r\n3. people want non-integer keys. For FAH-style projects, keying by (proj,run,clone) would be nice. Strings could be used to label trajectories \"apo\" or \"bound\".\r\n\r\n\r\nFor now, the scripts will include\r\n```python\r\nout = {}\r\nfor k, traj in trajs:\r\n  out[k] = estimator.partial_transform(traj)\r\n```\r\n\r\nBut it would be nice to just do\r\n```python\r\nout = estimator.transform(trajs)\r\n```"},"comment":{"url":"https://api.github.com/repos/msmbuilder/msmbuilder/issues/comments/206640237","html_url":"https://github.com/msmbuilder/msmbuilder/pull/768#issuecomment-206640237","issue_url":"https://api.github.com/repos/msmbuilder/msmbuilder/issues/768","id":206640237,"user":{"login":"msultan","id":3597136,"avatar_url":"https://avatars.githubusercontent.com/u/3597136?v=3","gravatar_id":"","url":"https://api.github.com/users/msultan","html_url":"https://github.com/msultan","followers_url":"https://api.github.com/users/msultan/followers","following_url":"https://api.github.com/users/msultan/following{/other_user}","gists_url":"https://api.github.com/users/msultan/gists{/gist_id}","starred_url":"https://api.github.com/users/msultan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/msultan/subscriptions","organizations_url":"https://api.github.com/users/msultan/orgs","repos_url":"https://api.github.com/users/msultan/repos","events_url":"https://api.github.com/users/msultan/events{/privacy}","received_events_url":"https://api.github.com/users/msultan/received_events","type":"User","site_admin":false},"created_at":"2016-04-07T01:03:50Z","updated_at":"2016-04-07T01:03:50Z","body":"i keep getting errors when i try to run this. I think i might be missing something. \r\n\r\n``` bash\r\nmsmb ProjectTemplate --flavor generic\r\nAn unexpected error has occurred with MSMBuilder (version 3.5.0.dev0.dev-58a13dc), please\r\nconsider sending the following traceback to MSMBuilder GitHub issue tracker at:\r\n            https://github.com/msmbuilder/msmbuilder/issues\r\n\r\nTraceback (most recent call last):\r\n  File \"/Users/muneeb/anaconda/bin/msmb\", line 9, in <module>\r\n    load_entry_point('msmbuilder==3.5.0.dev0', 'console_scripts', 'msmb')()\r\n  File \"/Users/muneeb/anaconda/lib/python3.5/site-packages/msmbuilder-3.5.0.dev0-py3.5-macosx-10.5-x86_64.egg/msmbuilder/scripts/msmb.py\", line 28, in main\r\n    app.start()\r\n  File \"/Users/muneeb/anaconda/lib/python3.5/site-packages/msmbuilder-3.5.0.dev0-py3.5-macosx-10.5-x86_64.egg/msmbuilder/cmdline.py\", line 464, in start\r\n    instance.start()\r\n  File \"/Users/muneeb/anaconda/lib/python3.5/site-packages/msmbuilder-3.5.0.dev0-py3.5-macosx-10.5-x86_64.egg/msmbuilder/commands/project.py\", line 31, in start\r\n    self.instance.write_all()\r\n  File \"/Users/muneeb/anaconda/lib/python3.5/site-packages/msmbuilder-3.5.0.dev0-py3.5-macosx-10.5-x86_64.egg/msmbuilder/io.py\", line 392, in write_all\r\n    for k, ext, rendered in self.render_iter():\r\n  File \"/Users/muneeb/anaconda/lib/python3.5/site-packages/msmbuilder-3.5.0.dev0-py3.5-macosx-10.5-x86_64.egg/msmbuilder/io.py\", line 376, in render_iter\r\n    templ_dict = self.get_templates(env.list_templates())\r\n  File \"/Users/muneeb/anaconda/lib/python3.5/site-packages/jinja2/environment.py\", line 723, in list_templates\r\n    x = self.loader.list_templates()\r\n  File \"/Users/muneeb/anaconda/lib/python3.5/site-packages/jinja2/loaders.py\", line 265, in list_templates\r\n    _walk(path)\r\n  File \"/Users/muneeb/anaconda/lib/python3.5/site-packages/jinja2/loaders.py\", line 259, in _walk\r\n    for filename in self.provider.resource_listdir(path):\r\n  File \"/Users/muneeb/anaconda/lib/python3.5/site-packages/setuptools-20.3-py3.5.egg/pkg_resources/__init__.py\", line 1466, in resource_listdir\r\n  File \"/Users/muneeb/anaconda/lib/python3.5/site-packages/setuptools-20.3-py3.5.egg/pkg_resources/__init__.py\", line 1554, in _listdir\r\nFileNotFoundError: [Errno 2] No such file or directory: '/Users/muneeb/anaconda/lib/python3.5/site-packages/msmbuilder-3.5.0.dev0-py3.5-macosx-10.5-x86_64.egg/msmbuilder/project_templates'\r\n\r\n```"}}