{"action":"created","issue":{"url":"https://api.github.com/repos/rundeck/rundeck/issues/1982","repository_url":"https://api.github.com/repos/rundeck/rundeck","labels_url":"https://api.github.com/repos/rundeck/rundeck/issues/1982/labels{/name}","comments_url":"https://api.github.com/repos/rundeck/rundeck/issues/1982/comments","events_url":"https://api.github.com/repos/rundeck/rundeck/issues/1982/events","html_url":"https://github.com/rundeck/rundeck/issues/1982","id":169353454,"number":1982,"title":"SCM Import/Export with high job count","user":{"login":"penekk","id":12582893,"avatar_url":"https://avatars.githubusercontent.com/u/12582893?v=3","gravatar_id":"","url":"https://api.github.com/users/penekk","html_url":"https://github.com/penekk","followers_url":"https://api.github.com/users/penekk/followers","following_url":"https://api.github.com/users/penekk/following{/other_user}","gists_url":"https://api.github.com/users/penekk/gists{/gist_id}","starred_url":"https://api.github.com/users/penekk/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/penekk/subscriptions","organizations_url":"https://api.github.com/users/penekk/orgs","repos_url":"https://api.github.com/users/penekk/repos","events_url":"https://api.github.com/users/penekk/events{/privacy}","received_events_url":"https://api.github.com/users/penekk/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2016-08-04T11:34:40Z","updated_at":"2016-08-30T09:36:20Z","closed_at":null,"body":"With versions 2.6.6 and 2.6.8 configuring SCM export for a substantially sized project ( ~ 750 jobs ) results in awfully long processing time (from my understanding its generating files in local repository) - nginx as proxy times out and the process continues in the background. Moreover, during that time accessing Job list of the project resulted in \"Cannot serialize job\" errors, until all of the files were generated. Scheduled jobs however work fine.\r\nOnce I discovered https://github.com/variacode95/rundeck/commit/0d805387196924405368bcd0e554226590777424\r\nand https://github.com/variacode95/rundeck/commit/52674c9a223c86a9600872ca7b2dbbb18374f47b) I decided to upgrade to 2.6.9 a moment ago. No serialize error has occured yet , although the process still takes ages.\r\n\r\nConfiguring the Import however, results in:\r\n\r\n```\r\nRequest: /project/[project]/jobs?max=20&offset=0\r\nMessage: Cannot get property 'tree' on null object \r\nCaused by: Cannot get property 'tree' on null object \r\nClass: MenuController \r\nAt Line: [231] \r\nCode Snippet:\r\nStack Trace\r\n\r\njava.lang.NullPointerException: Cannot get property 'tree' on null object\r\n\tat org.rundeck.plugin.scm.git.GitImportPlugin.importSynchStateForStatus(GitImportPlugin.groovy:302)\r\n\tat org.rundeck.plugin.scm.git.GitImportPlugin.refreshJobStatus(GitImportPlugin.groovy:242)\r\n\tat org.rundeck.plugin.scm.git.GitImportPlugin.getJobStatus(GitImportPlugin.groovy:349)\r\n\tat org.rundeck.plugin.scm.git.GitImportPlugin.getJobStatus(GitImportPlugin.groovy:337)\r\n\tat rundeck.services.ScmService$_importStatusForJobs_closure15.doCall(ScmService.groovy:961)\r\n\tat rundeck.services.ScmService.importStatusForJobs(ScmService.groovy:956)\r\n\tat rundeck.controllers.MenuController$_closure10.doCall(MenuController.groovy:323)\r\n\tat rundeck.controllers.MenuController$_closure9.doCall(MenuController.groovy:231)\r\n\tat org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:565)\r\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1360)\r\n\tat grails.plugin.cache.web.filter.PageFragmentCachingFilter.doFilter(PageFragmentCachingFilter.java:198)\r\n\tat grails.plugin.cache.web.filter.AbstractFilter.doFilter(AbstractFilter.java:63)\r\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1331)\r\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1331)\r\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1331)\r\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1331)\r\n\tat org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:477)\r\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)\r\n\tat org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:539)\r\n\tat org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:227)\r\n\tat org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1031)\r\n\tat org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:406)\r\n\tat org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:186)\r\n\tat org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:965)\r\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)\r\n\tat org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:288)\r\n\tat org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:115)\r\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1331)\r\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1331)\r\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1331)\r\n\tat com.codahale.metrics.servlet.AbstractInstrumentedFilter.doFilter(AbstractInstrumentedFilter.java:97)\r\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1331)\r\n\tat com.dtolabs.rundeck.server.filters.AuthFilter.doFilter(AuthFilter.java:58)\r\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1331)\r\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1331)\r\n\tat org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:477)\r\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)\r\n\tat org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:486)\r\n\tat org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:227)\r\n\tat org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1031)\r\n\tat org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:406)\r\n\tat org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:186)\r\n\tat org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:965)\r\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)\r\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111)\r\n\tat org.eclipse.jetty.server.Server.handle(Server.java:349)\r\n\tat org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:449)\r\n\tat org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:910)\r\n\tat org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:634)\r\n\tat org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:230)\r\n\tat org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:76)\r\n\tat org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:609)\r\n\tat org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:45)\r\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)\r\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)\r\n\tat java.lang.Thread.run(Thread.java:745)\r\n```\r\n\r\nRepeating configuration for a much much smaller project (31 jobs) seems to be working okay, at least during few brief tests, although for the jobs to start being tracked (import) I have to restart rundeck entirely - after that it both detects changes made from rundeck ui and in remote repo, which then can be pushed/pulled respectively.\r\n\r\nHas anyone tested/investigated this? Current state makes the plugin unsusable for a project of this size which at the moment is a crucial feature I am trying to get to work. I am also wondering, whether having tried configuring the project multiple times is the reason the 'null tree' error persists. Mariadb datasource and rundeck instance(s) hosts are located in the same dc.\r\n\r\nEdit: perhaps this may be of relevance, there are 2 rundeck instances connected to the same database, one is running in passive mode as slave, with cluster mode disabled (takeover is via vrrp)."},"comment":{"url":"https://api.github.com/repos/rundeck/rundeck/issues/comments/243386214","html_url":"https://github.com/rundeck/rundeck/issues/1982#issuecomment-243386214","issue_url":"https://api.github.com/repos/rundeck/rundeck/issues/1982","id":243386214,"user":{"login":"penekk","id":12582893,"avatar_url":"https://avatars.githubusercontent.com/u/12582893?v=3","gravatar_id":"","url":"https://api.github.com/users/penekk","html_url":"https://github.com/penekk","followers_url":"https://api.github.com/users/penekk/followers","following_url":"https://api.github.com/users/penekk/following{/other_user}","gists_url":"https://api.github.com/users/penekk/gists{/gist_id}","starred_url":"https://api.github.com/users/penekk/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/penekk/subscriptions","organizations_url":"https://api.github.com/users/penekk/orgs","repos_url":"https://api.github.com/users/penekk/repos","events_url":"https://api.github.com/users/penekk/events{/privacy}","received_events_url":"https://api.github.com/users/penekk/received_events","type":"User","site_admin":false},"created_at":"2016-08-30T09:36:20Z","updated_at":"2016-08-30T09:36:20Z","body":"During that time new errors haven't occured. The loading times are still extremely long, which renders rundeck webui close to excruciating sometimes."}}
{"actor":{"display_login":"penekk"}}