{"action":"created","issue":{"url":"https://api.github.com/repos/johnmyleswhite/ProjectTemplate/issues/152","repository_url":"https://api.github.com/repos/johnmyleswhite/ProjectTemplate","labels_url":"https://api.github.com/repos/johnmyleswhite/ProjectTemplate/issues/152/labels{/name}","comments_url":"https://api.github.com/repos/johnmyleswhite/ProjectTemplate/issues/152/comments","events_url":"https://api.github.com/repos/johnmyleswhite/ProjectTemplate/issues/152/events","html_url":"https://github.com/johnmyleswhite/ProjectTemplate/pull/152","id":176173789,"number":152,"title":"Some updates to caching and config","user":{"login":"connectedblue","id":16526825,"avatar_url":"https://avatars.githubusercontent.com/u/16526825?v=3","gravatar_id":"","url":"https://api.github.com/users/connectedblue","html_url":"https://github.com/connectedblue","followers_url":"https://api.github.com/users/connectedblue/followers","following_url":"https://api.github.com/users/connectedblue/following{/other_user}","gists_url":"https://api.github.com/users/connectedblue/gists{/gist_id}","starred_url":"https://api.github.com/users/connectedblue/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/connectedblue/subscriptions","organizations_url":"https://api.github.com/users/connectedblue/orgs","repos_url":"https://api.github.com/users/connectedblue/repos","events_url":"https://api.github.com/users/connectedblue/events{/privacy}","received_events_url":"https://api.github.com/users/connectedblue/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":11,"created_at":"2016-09-10T10:50:29Z","updated_at":"2016-09-14T21:33:05Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/johnmyleswhite/ProjectTemplate/pulls/152","html_url":"https://github.com/johnmyleswhite/ProjectTemplate/pull/152","diff_url":"https://github.com/johnmyleswhite/ProjectTemplate/pull/152.diff","patch_url":"https://github.com/johnmyleswhite/ProjectTemplate/pull/152.patch"},"body":"Hi John\r\n\r\nI love using Project Template.   I love how it brings clarity to the analysis process.\r\n\r\nFor a while I've been creating some custom functions in lib to help certain projects, and I've been cut and pasting them into each project which is just horrible.  So I created a fork of the package to store the functions in my own version of ProjectTemplate.\r\n\r\nI thought I'd share them with you to see if they may be useful to put in the core package.  The functions I added are:\r\n\r\n- update cache only if there's not already a cache object (but do default behaviour of caching always).  This allows munge scripts to skip caching if it takes a long time which allows load.project to run quicker.\r\n\r\n- have a file called custom.dcf in the config directory which adds config variables which are relevant for that project only.  These can be anything (char, numeric, logical) etc and allows for analysis configuration (e.g. I use it to determine which variables should be cached in the analysis)\r\n\r\n- new function clear.cache which allows specific cache objects to be deleted (forcing data to be re-read from raw data).  I also use this to only run long analysis in munge if the data isn't already cached which saves time again when load.project is run.\r\n\r\nThe overall theme of the changes are about trying to write munge processing in a way that executes quickly when load.project is run.  If you are building up specific dataset transformations on the raw data, you can cache them if they take a long time to run.  The munge logic can then be written to only execute if the object doesn't already exist in the global environment\r\n\r\nIf however your data changes, or you update the transformation logic, then you simply clear the cache and run load.project again.\r\n\r\nAppreciate this is a long note, but would appreciate feedback on whether you feel this functionality would be useful to develop into the core package.\r\n\r\nbest regards\r\nChris\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/johnmyleswhite/ProjectTemplate/issues/comments/247161924","html_url":"https://github.com/johnmyleswhite/ProjectTemplate/pull/152#issuecomment-247161924","issue_url":"https://api.github.com/repos/johnmyleswhite/ProjectTemplate/issues/152","id":247161924,"user":{"login":"connectedblue","id":16526825,"avatar_url":"https://avatars.githubusercontent.com/u/16526825?v=3","gravatar_id":"","url":"https://api.github.com/users/connectedblue","html_url":"https://github.com/connectedblue","followers_url":"https://api.github.com/users/connectedblue/followers","following_url":"https://api.github.com/users/connectedblue/following{/other_user}","gists_url":"https://api.github.com/users/connectedblue/gists{/gist_id}","starred_url":"https://api.github.com/users/connectedblue/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/connectedblue/subscriptions","organizations_url":"https://api.github.com/users/connectedblue/orgs","repos_url":"https://api.github.com/users/connectedblue/repos","events_url":"https://api.github.com/users/connectedblue/events{/privacy}","received_events_url":"https://api.github.com/users/connectedblue/received_events","type":"User","site_admin":false},"created_at":"2016-09-14T21:33:05Z","updated_at":"2016-09-14T21:33:05Z","body":"Yes, if it's acceptable to encode convention, then we should allow `load.project` to do magic in that case.\r\n\r\nSmall patch added to the branch to achieve this, which I've tested on one of my more complex lib directories.\r\n\r\n"}}
{"actor":{"display_login":"connectedblue"}}