{"action":"created","issue":{"url":"https://api.github.com/repos/Burke-Lauenroth-Lab/SoilWat_R_Wrapper/issues/52","repository_url":"https://api.github.com/repos/Burke-Lauenroth-Lab/SoilWat_R_Wrapper","labels_url":"https://api.github.com/repos/Burke-Lauenroth-Lab/SoilWat_R_Wrapper/issues/52/labels{/name}","comments_url":"https://api.github.com/repos/Burke-Lauenroth-Lab/SoilWat_R_Wrapper/issues/52/comments","events_url":"https://api.github.com/repos/Burke-Lauenroth-Lab/SoilWat_R_Wrapper/issues/52/events","html_url":"https://github.com/Burke-Lauenroth-Lab/SoilWat_R_Wrapper/issues/52","id":191231202,"number":52,"title":"do_OneSite() calls have large memory footprint when run with many climate conditions","user":{"login":"dschlaep","id":4205019,"avatar_url":"https://avatars.githubusercontent.com/u/4205019?v=3","gravatar_id":"","url":"https://api.github.com/users/dschlaep","html_url":"https://github.com/dschlaep","followers_url":"https://api.github.com/users/dschlaep/followers","following_url":"https://api.github.com/users/dschlaep/following{/other_user}","gists_url":"https://api.github.com/users/dschlaep/gists{/gist_id}","starred_url":"https://api.github.com/users/dschlaep/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dschlaep/subscriptions","organizations_url":"https://api.github.com/users/dschlaep/orgs","repos_url":"https://api.github.com/users/dschlaep/repos","events_url":"https://api.github.com/users/dschlaep/events{/privacy}","received_events_url":"https://api.github.com/users/dschlaep/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2016-11-23T10:14:15Z","updated_at":"2016-11-23T10:42:40Z","closed_at":null,"body":"Workers for on a project with 45 climate conditions use 2-3.5 GB memory. (Why the large spread?)\r\n\r\nThe local variables of `do_OneSite()` which scale with number of climate conditions (`scenario_No`) are\r\n- input variables: `swRunScenariosData`, `i_sw_weatherList`, `grasses.c3c4ann.fractions`\r\n- output variable: `runData`, `SQL`\r\n\r\nIf the main culprits are among the input variables, then we may need to consider to call `do_OneSite()` for each `P_id` instead for each `i_sim` (and pass `swRunScenariosData[[1]]` as default to the calls for the scenarios.\r\nIf the main culprits are among the output variables, then we may need to consider to combine the 'execute' and 'aggregate' sections, and write `SQL` to file after each scenario."},"comment":{"url":"https://api.github.com/repos/Burke-Lauenroth-Lab/SoilWat_R_Wrapper/issues/comments/262481361","html_url":"https://github.com/Burke-Lauenroth-Lab/SoilWat_R_Wrapper/issues/52#issuecomment-262481361","issue_url":"https://api.github.com/repos/Burke-Lauenroth-Lab/SoilWat_R_Wrapper/issues/52","id":262481361,"user":{"login":"dschlaep","id":4205019,"avatar_url":"https://avatars.githubusercontent.com/u/4205019?v=3","gravatar_id":"","url":"https://api.github.com/users/dschlaep","html_url":"https://github.com/dschlaep","followers_url":"https://api.github.com/users/dschlaep/followers","following_url":"https://api.github.com/users/dschlaep/following{/other_user}","gists_url":"https://api.github.com/users/dschlaep/gists{/gist_id}","starred_url":"https://api.github.com/users/dschlaep/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dschlaep/subscriptions","organizations_url":"https://api.github.com/users/dschlaep/orgs","repos_url":"https://api.github.com/users/dschlaep/repos","events_url":"https://api.github.com/users/dschlaep/events{/privacy}","received_events_url":"https://api.github.com/users/dschlaep/received_events","type":"User","site_admin":false},"created_at":"2016-11-23T10:42:40Z","updated_at":"2016-11-23T10:42:40Z","body":"R object sizes for my project with 45 climate conditions and 313 output fields using `print(object.size(var), unit = \"Mb\")` for one specific run:\r\n- `swRunScenariosData`: current scenario = 0.1 Mb; all 45 scenarios = 2.3 Mb\r\n- `i_sw_weatherList`: current scenario = 0.4 Mb; all 45 scenarios = 18 Mb\r\n- `grasses.c3c4ann.fractions`: current scenario = 0.1 Kb; all 45 scenarios = 3.6 Kb\r\n- `runData`: current scenario = 23.7 Mb; all 45 scenarios = 1065 Mb\r\n- `SQL`: all 45 scenarios = 0.4 Mb\r\n\r\n==> `runData` is by far the largest variable\r\nI suggest to combine the 'execute' and 'aggregate' sections into the same loop so that `runData` only ever contains SOILWAT output from one scenario/run"}}
{"actor":{"display_login":"dschlaep"}}