{"action":"created","issue":{"url":"https://api.github.com/repos/Mouse-Imaging-Centre/RMINC/issues/99","repository_url":"https://api.github.com/repos/Mouse-Imaging-Centre/RMINC","labels_url":"https://api.github.com/repos/Mouse-Imaging-Centre/RMINC/issues/99/labels{/name}","comments_url":"https://api.github.com/repos/Mouse-Imaging-Centre/RMINC/issues/99/comments","events_url":"https://api.github.com/repos/Mouse-Imaging-Centre/RMINC/issues/99/events","html_url":"https://github.com/Mouse-Imaging-Centre/RMINC/issues/99","id":174609250,"number":99,"title":"RMINC 1.4.2.0 mincLmer throws subscript out of bounds errors with SGE parallelization","user":{"login":"gdevenyi","id":3001850,"avatar_url":"https://avatars.githubusercontent.com/u/3001850?v=3","gravatar_id":"","url":"https://api.github.com/users/gdevenyi","html_url":"https://github.com/gdevenyi","followers_url":"https://api.github.com/users/gdevenyi/followers","following_url":"https://api.github.com/users/gdevenyi/following{/other_user}","gists_url":"https://api.github.com/users/gdevenyi/gists{/gist_id}","starred_url":"https://api.github.com/users/gdevenyi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gdevenyi/subscriptions","organizations_url":"https://api.github.com/users/gdevenyi/orgs","repos_url":"https://api.github.com/users/gdevenyi/repos","events_url":"https://api.github.com/users/gdevenyi/events{/privacy}","received_events_url":"https://api.github.com/users/gdevenyi/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":6,"created_at":"2016-09-01T19:46:00Z","updated_at":"2016-10-18T15:40:54Z","closed_at":null,"body":"```R\r\n> library(RMINC)\r\nLoading required package: BatchJobs\r\nLoading required package: BBmisc\r\nSourcing configuration file: '/opt/quarantine/R-extras/3.2.4/build/BatchJobs/etc/BatchJobs_global_config.R'\r\nBatchJobs configuration:\r\n  cluster functions: SGE\r\n  mail.from: \r\n  mail.to: \r\n  mail.start: none\r\n  mail.done: none\r\n  mail.error: none\r\n  default.resources: \r\n  debug: FALSE\r\n  raise.warnings: FALSE\r\n  staged.queries: TRUE\r\n  max.concurrent.jobs: Inf\r\n  fs.timeout: NA\r\n```\r\n\r\n```\r\n> cat /opt/quarantine/R-extras/3.2.4/build/BatchJobs/etc/BatchJobs_global_config.R\r\ncluster.functions = makeClusterFunctionsSGE(\"/opt/quarantine/R-extras/3.2.4/build/BatchJobs/etc/sge_script.tmpl\")\r\nmail.start = \"none\"\r\nmail.done = \"none\"\r\nmail.error = \"none\"\r\ndb.driver = \"SQLite\"\r\ndb.options = list()\r\n```\r\n\r\n```\r\n> cat /opt/quarantine/R-extras/3.2.4/build/BatchJobs/etc/sge_script.tmpl\r\n#!/bin/bash\r\n\r\n# The name of the job, can be anything, simply used when displaying the list of running jobs\r\n#$ -N <%= job.name %>\r\n# Combining output/error messages into one file\r\n#$ -j y\r\n# Giving the name of the output log file\r\n#$ -o <%= log.file %>\r\n# One needs to tell the queue system to use the current directory as the working directory\r\n# Or else the script may fail as it will execute in your top level home directory /home/username\r\n#$ -cwd\r\n# use environment variables\r\n#$ -V\r\n##### use correct queue \r\n#####$ -q <%= resources$queue %>\r\n##### use job arrays\r\n#####$ -t 1-<%= arrayjobs %>\r\n\r\n# we merge R output with stdout from SGE, which gets then logged via -o option\r\nR CMD BATCH --no-save --no-restore \"<%= rscript %>\" /dev/stdout\r\nexit 0\r\n\r\n```\r\n\r\n```R\r\n> cheese<-mincLmer(reljac02 ~ group*timepoint*age + sex + (1|subject), squeak, ask=\"/opt/quarantine/resources/DorrSteadmanUllmann/MBM_100um/Dorr_2008_Steadman_2013_Ullmann_2013_on_NRXN1a_v1_mask.mnc\", parallel = c(\"sge\", 240))\r\nError in results[[1]] : subscript out of bounds\r\nError in removeRegistry(qMinc_registry, ask = \"no\") : \r\n  Can't remove registry, because there are 1 live jobs on the system.\r\n```"},"comment":{"url":"https://api.github.com/repos/Mouse-Imaging-Centre/RMINC/issues/comments/254548368","html_url":"https://github.com/Mouse-Imaging-Centre/RMINC/issues/99#issuecomment-254548368","issue_url":"https://api.github.com/repos/Mouse-Imaging-Centre/RMINC/issues/99","id":254548368,"user":{"login":"cfhammill","id":7467038,"avatar_url":"https://avatars.githubusercontent.com/u/7467038?v=3","gravatar_id":"","url":"https://api.github.com/users/cfhammill","html_url":"https://github.com/cfhammill","followers_url":"https://api.github.com/users/cfhammill/followers","following_url":"https://api.github.com/users/cfhammill/following{/other_user}","gists_url":"https://api.github.com/users/cfhammill/gists{/gist_id}","starred_url":"https://api.github.com/users/cfhammill/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cfhammill/subscriptions","organizations_url":"https://api.github.com/users/cfhammill/orgs","repos_url":"https://api.github.com/users/cfhammill/repos","events_url":"https://api.github.com/users/cfhammill/events{/privacy}","received_events_url":"https://api.github.com/users/cfhammill/received_events","type":"User","site_admin":false},"created_at":"2016-10-18T15:40:54Z","updated_at":"2016-10-18T15:40:54Z","body":"Hi Gabe, \r\n\r\nThere are substantial changes coming to parallelism coming with the next release (toward the end of the week it's looking like). RMINC no longer tries to guess and set reasonable default parameters for things like vmem. It comes with a new function `configureMincParallel` which is a thin wrapper to `BatchJobs::loadConfig` except it comes with a defaults for PBS and SGE. These both request 8G by default now, but it's trivial to adjust this either through `BatchJobs::setConfig` (or `loadConfig`) directly or with configureMincParallel. I've tried my best to document the new system but if you have any trouble getting it to do what you want let me know.\r\n\r\nAs for estimating the amount of memory necessary, I'm unsure if it's useful enough to break down the nice barrier between configuration and execution. We could potentially provide a lookup table of n-voxels x m-files with estimates of required memory, and these can be used with `setConfig`.\r\n\r\nNot sure what the rest of the devs think."}}
{"actor":{"display_login":"cfhammill"}}