{"action":"created","issue":{"url":"https://api.github.com/repos/irods/irods/issues/3133","repository_url":"https://api.github.com/repos/irods/irods","labels_url":"https://api.github.com/repos/irods/irods/issues/3133/labels{/name}","comments_url":"https://api.github.com/repos/irods/irods/issues/3133/comments","events_url":"https://api.github.com/repos/irods/irods/issues/3133/events","html_url":"https://github.com/irods/irods/issues/3133","id":146369825,"number":3133,"title":"Unable to start iRODS resource server","user":{"login":"dcmss","id":5564662,"avatar_url":"https://avatars.githubusercontent.com/u/5564662?v=3","gravatar_id":"","url":"https://api.github.com/users/dcmss","html_url":"https://github.com/dcmss","followers_url":"https://api.github.com/users/dcmss/followers","following_url":"https://api.github.com/users/dcmss/following{/other_user}","gists_url":"https://api.github.com/users/dcmss/gists{/gist_id}","starred_url":"https://api.github.com/users/dcmss/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dcmss/subscriptions","organizations_url":"https://api.github.com/users/dcmss/orgs","repos_url":"https://api.github.com/users/dcmss/repos","events_url":"https://api.github.com/users/dcmss/events{/privacy}","received_events_url":"https://api.github.com/users/dcmss/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":6,"created_at":"2016-04-06T16:42:52Z","updated_at":"2016-04-06T22:49:01Z","closed_at":null,"body":"Here is the output from the log. Appreciate any help!\r\n\r\n```\r\nApr  6 10:15:22 pid:3922 NOTICE:    RescName: dsnetresource, VaultPath: /tmp/dsnet/vault\r\nApr  6 10:15:22 pid:3922 NOTICE:    RescName: bundleResc, VaultPath: /bundle\r\nZone Info:\r\n    ZoneName: tempZone   Type: LOCAL_ICAT    HostAddr: fr-s-hpcdm-gp-d.ncifcrf.gov   PortNum: 1247\r\n\r\nreHost:   fr-s-hpcdm-gp-d.ncifcrf.gov\r\n\r\nApr  6 10:15:22 pid:3922 NOTICE: rodsServer Release version rods4.1.8 - API Version d is up\r\nApr  6 10:15:22 pid:3922 NOTICE: >>> control plane :: listening on port 1248\r\nApr  6 10:15:27 pid:3922 NOTICE: Agent process 3938 started for puser=rods and cuser=rods from 129.43.165.62\r\nApr  6 10:15:28 pid:3938 NOTICE: readAndProcClientMsg: received disconnect msg from client\r\nApr  6 10:15:28 pid:3938 NOTICE: Agent exiting with status = 0\r\nApr  6 10:15:28 pid:3922 NOTICE: Agent process 3938 exited with status 0\r\nApr  6 10:15:28 pid:3922 NOTICE: Agent process 3940 started for puser=rods and cuser=rods from 129.43.165.62\r\nApr  6 10:15:28 pid:3940 NOTICE: readAndProcClientMsg: received disconnect msg from client\r\nApr  6 10:15:28 pid:3940 NOTICE: Agent exiting with status = 0\r\nApr  6 10:15:28 pid:3922 NOTICE: Agent process 3940 exited with status 0\r\nApr  6 10:15:28 pid:3922 NOTICE: Agent process 3943 started for puser=rods and cuser=rods from 129.43.165.62\r\nApr  6 10:15:28 pid:3943 NOTICE: readAndProcClientMsg: received disconnect msg from client\r\nApr  6 10:15:28 pid:3943 NOTICE: Agent exiting with status = 0\r\nApr  6 10:15:28 pid:3922 NOTICE: Agent process 3943 exited with status 0\r\nApr  6 10:15:28 pid:3922 NOTICE: Agent process 3945 started for puser=rods and cuser=rods from 129.43.165.62\r\nApr  6 10:15:29 pid:3922 NOTICE: rodsServer caught signal 2, exiting\r\nirodsServer: /home/irodsbuild/irods/external/boost_1_58_0z/boost/thread/pthread/condition_variable_fwd.hpp:81: boost::condition_variable::~condition_variable(): Assertion `!ret' failed.\r\n    LocalHostName:  localhost, fr-s-hpcdm-gp-p, fr-s-hpcdm-gp-p.ncifcrf.gov, Port Num: 1247.\r\n\r\nApr  6 10:31:21 pid:4130 NOTICE: rsGenQuery error. RCAT is not configured on this host\r\nApr  6 10:31:21 pid:4130 NOTICE: initZone: rsGenQuery error, status = -43000\r\nApr  6 10:31:21 pid:4130 SYSTEM FATAL: initServerInfo: initZone error, status = -43000\r\nApr  6 10:31:21 pid:4130 NOTICE: initServer: initServerInfo error, status = -43000\r\nApr  6 10:31:21 pid:4130 ERROR: initServerMain: initServer error. status = -43000\r\n    LocalHostName:  localhost, fr-s-hpcdm-gp-p, fr-s-hpcdm-gp-p.ncifcrf.gov, Port Num: 1247.\r\n\r\nApr  6 10:32:00 pid:4214 NOTICE: rsGenQuery error. RCAT is not configured on this host\r\nApr  6 10:32:00 pid:4214 NOTICE: initZone: rsGenQuery error, status = -43000\r\nApr  6 10:32:00 pid:4214 SYSTEM FATAL: initServerInfo: initZone error, status = -43000\r\nApr  6 10:32:00 pid:4214 NOTICE: initServer: initServerInfo error, status = -43000\r\nApr  6 10:32:00 pid:4214 ERROR: initServerMain: initServer error. status = -43000\r\n    LocalHostName:  localhost, fr-s-hpcdm-gp-p, fr-s-hpcdm-gp-p.ncifcrf.gov, Port Num: 1247.\r\n\r\nApr  6 12:27:58 pid:6171 NOTICE: rsGenQuery error. RCAT is not configured on this host\r\nApr  6 12:27:58 pid:6171 NOTICE: initZone: rsGenQuery error, status = -43000\r\nApr  6 12:27:58 pid:6171 SYSTEM FATAL: initServerInfo: initZone error, status = -43000\r\nApr  6 12:27:58 pid:6171 NOTICE: initServer: initServerInfo error, status = -43000\r\nApr  6 12:27:58 pid:6171 ERROR: initServerMain: initServer error. status = -43000\r\n    LocalHostName:  localhost, fr-s-hpcdm-gp-p, fr-s-hpcdm-gp-p.ncifcrf.gov, Port Num: 1247.\r\n\r\nApr  6 12:40:09 pid:6402 NOTICE:    RescName: dsnetresource, VaultPath: /tmp/dsnet/vault\r\nApr  6 12:40:09 pid:6402 NOTICE:    RescName: bundleResc, VaultPath: /bundle\r\nZone Info:\r\n    ZoneName: tempZone   Type: LOCAL_ICAT    HostAddr: fr-s-hpcdm-gp-d   PortNum: 1247\r\n\r\nreHost:   fr-s-hpcdm-gp-d\r\n\r\nApr  6 12:40:09 pid:6402 NOTICE: rodsServer Release version rods4.1.8 - API Version d is up\r\nApr  6 12:40:09 pid:6402 NOTICE: >>> control plane :: listening on port 1248\r\nApr  6 12:40:15 pid:6402 NOTICE: Agent process 6418 started for puser=rods and cuser=rods from 129.43.165.62\r\nApr  6 12:40:15 pid:6418 NOTICE: readAndProcClientMsg: received disconnect msg from client\r\nApr  6 12:40:15 pid:6418 NOTICE: Agent exiting with status = 0\r\nApr  6 12:40:15 pid:6402 NOTICE: Agent process 6418 exited with status 0\r\nApr  6 12:40:15 pid:6402 NOTICE: Agent process 6420 started for puser=rods and cuser=rods from 129.43.165.62\r\nApr  6 12:40:15 pid:6420 NOTICE: readAndProcClientMsg: received disconnect msg from client\r\nApr  6 12:40:15 pid:6420 NOTICE: Agent exiting with status = 0\r\nApr  6 12:40:15 pid:6402 NOTICE: Agent process 6420 exited with status 0\r\nApr  6 12:40:15 pid:6402 NOTICE: Agent process 6422 started for puser=rods and cuser=rods from 129.43.165.62\r\nApr  6 12:40:15 pid:6402 NOTICE: rodsServer caught signal 2, exiting\r\nirodsServer: /home/irodsbuild/irods/external/boost_1_58_0z/boost/thread/pthread/condition_variable_fwd.hpp:81: boost::condition_variable::~condition_variable(): Assertion `!ret' failed.\r\n\r\n\r\n\r\n++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++\r\n\r\nHere is the setup log:\r\n\r\niRODS setup\r\n------------------------------------------------------------------------\r\nWed Apr  6 12:40:04 2016\r\n\r\nScript:\r\n    Script:  ./scripts/perl/irods_setup.pl\r\n    CWD:  /var/lib/irods/iRODS\r\n\r\nPerl:\r\n    Perl path:  /usr/bin/perl\r\n    Perl version:  5.018002\r\n\r\nSystem:\r\n    OS:  Linux\r\n    User:  irods\r\n    User ID:  999\r\n    Host name:  fr-s-hpcdm-gp-p.ncifcrf.gov\r\n    Host addresses:  129.43.165.62 127.0.0.1\r\n\r\nEnvironment:\r\n    LOGNAME = irods\r\n    OLDPWD = /var/lib/irods\r\n    SERVICE_MIX_HOME = /opt/apache-servicemix-5.4.0\r\n    LANG = en_US.UTF-8\r\n    USER = irods\r\n    TERM = xterm\r\n    JAVA_HOME = /usr/lib/jvm/java-7-openjdk-amd64\r\n    MAIL = /var/mail/irods\r\n    _ = /usr/bin/perl\r\n    PATH = /bin:/opt/apache-servicemix-5.4.0/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games\r\n    SHLVL = 2\r\n    PWD = /var/lib/irods/iRODS\r\n    HOME = /var/lib/irods\r\n    SHELL = /bin/bash\r\n\r\nScript versions:\r\n    utils_print.pl = Apr 2015\r\n    utils_paths.pl = January 2015\r\n    utils_config.pl = Feb 2015\r\n    utils_platform.pl = September 2011\r\n    irods_setup.pl = Feb 2015\r\n    utils_file.pl = September 2011\r\n    utils_prompt.pl = March 2010\r\n\r\nPreparing to initialize...\r\n------------------------------------------------------------------------\r\nWed Apr  6 12:40:08 2016\r\n\r\nStopping iRODS server...\r\n\r\nConfiguring iRODS server...\r\n------------------------------------------------------------------------\r\nWed Apr  6 12:40:08 2016\r\n\r\nUpdating /etc/irods/server_config.json...\r\n    icat_host        = fr-s-hpcdm-gp-d\r\n    zone_name        = tempZone\r\n    zone_port        = 1247\r\n    zone_user        = rods\r\n    zone_auth_scheme = native\r\nRunning 'iinit' to enable server to server connections...\r\nUsing ICAT-enabled server on 'fr-s-hpcdm-gp-d'\r\n\r\nConfiguring iRODS user and starting server...\r\n------------------------------------------------------------------------\r\nWed Apr  6 12:40:08 2016\r\n\r\nUpdating iRODS user's ~/.irods/irods_environment.json...\r\n    move /var/lib/irods/.irods/irods_environment.json /var/lib/irods/.irods/irods_environment.json.orig\r\n\r\nStarting iRODS server...\r\nValidating [/var/lib/irods/.irods/irods_environment.json]... Success\r\nValidating [/etc/irods/server_config.json]... Success\r\nValidating [/etc/irods/hosts_config.json]... Success\r\nValidating [/etc/irods/host_access_control_config.json]... Success\r\n\r\nOpening iRODS connection...\r\n\r\nCreating default resource...\r\n    Skipped.  Resource [fr-s-hpcdm-gp-pResource] already created.\r\n\r\nTesting resource...\r\n\r\nCannot put test file into iRODS:\r\n    ERROR: [-]  iRODS/lib/core/src/procApiRequest.cpp:266:readAndProcApiReply :  status [SYS_HEADER_READ_LEN_ERR]  errno [No such file or directory] -- message []\r\n        [-]     iRODS/lib/core/src/sockComm.cpp:199:readMsgHeader :  status [SYS_HEADER_READ_LEN_ERR]  errno [No such file or directory] -- message [failed to call 'read header']\r\n                [-]     libtcp.cpp:240:tcp_read_msg_header :  status [SYS_HEADER_READ_LEN_ERR]  errno [No such file or directory] -- message [read 0 expected 4]\r\n\r\n    ERROR: putUtil: put error for /tempZone/home/rods/irods_put_fr-s-hpcdm-gp-p.ncifcrf.gov.tmp, status = -4002 status = -4002 SYS_HEADER_READ_LEN_ERR, No such file or directory\r\n    ERROR: [-]  iRODS/lib/core/src/rcConnect.cpp:269:rcDisconnect :  status [SYS_HEADER_WRITE_LEN_ERR]  errno [Broken pipe] -- message []\r\n        [-]     iRODS/lib/core/src/sockComm.cpp:1394:sendRodsMsg :  status [SYS_HEADER_WRITE_LEN_ERR]  errno [Broken pipe] -- message [failed to call 'write body']\r\n                [-]     libtcp.cpp:420:tcp_send_rods_msg :  status [SYS_HEADER_WRITE_LEN_ERR]  errno [Broken pipe] -- message [writeMsgHeader failed]\r\n                        [-]     iRODS/lib/core/src/sockComm.cpp:473:writeMsgHeader :  status [SYS_HEADER_WRITE_LEN_ERR]  errno [Broken pipe] -- message []\r\n                                [-]     libtcp.cpp:358:tcp_write_msg_header :  status [SYS_HEADER_WRITE_LEN_ERR]  errno [Broken pipe] -- message [wrote 0 expected 140]\r\n\r\n\r\nAbort.\r\n```"},"comment":{"url":"https://api.github.com/repos/irods/irods/issues/comments/206602549","html_url":"https://github.com/irods/irods/issues/3133#issuecomment-206602549","issue_url":"https://api.github.com/repos/irods/irods/issues/3133","id":206602549,"user":{"login":"kellerb","id":7625444,"avatar_url":"https://avatars.githubusercontent.com/u/7625444?v=3","gravatar_id":"","url":"https://api.github.com/users/kellerb","html_url":"https://github.com/kellerb","followers_url":"https://api.github.com/users/kellerb/followers","following_url":"https://api.github.com/users/kellerb/following{/other_user}","gists_url":"https://api.github.com/users/kellerb/gists{/gist_id}","starred_url":"https://api.github.com/users/kellerb/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kellerb/subscriptions","organizations_url":"https://api.github.com/users/kellerb/orgs","repos_url":"https://api.github.com/users/kellerb/repos","events_url":"https://api.github.com/users/kellerb/events{/privacy}","received_events_url":"https://api.github.com/users/kellerb/received_events","type":"User","site_admin":false},"created_at":"2016-04-06T22:49:01Z","updated_at":"2016-04-06T22:49:01Z","body":"What is the output of running `hostname` on both the resource server and the icat server?"}}