{"action":"created","issue":{"url":"https://api.github.com/repos/xcat2/xcat-core/issues/5506","repository_url":"https://api.github.com/repos/xcat2/xcat-core","labels_url":"https://api.github.com/repos/xcat2/xcat-core/issues/5506/labels{/name}","comments_url":"https://api.github.com/repos/xcat2/xcat-core/issues/5506/comments","events_url":"https://api.github.com/repos/xcat2/xcat-core/issues/5506/events","html_url":"https://github.com/xcat2/xcat-core/issues/5506","id":350147579,"node_id":"MDU6SXNzdWUzNTAxNDc1Nzk=","number":5506,"title":"[ xCAT 2.14.2 Centos 7.5 ] Unable to get IP for OPA  interface in compute","user":{"login":"dspatil7768","id":22992849,"node_id":"MDQ6VXNlcjIyOTkyODQ5","avatar_url":"https://avatars2.githubusercontent.com/u/22992849?v=4","gravatar_id":"","url":"https://api.github.com/users/dspatil7768","html_url":"https://github.com/dspatil7768","followers_url":"https://api.github.com/users/dspatil7768/followers","following_url":"https://api.github.com/users/dspatil7768/following{/other_user}","gists_url":"https://api.github.com/users/dspatil7768/gists{/gist_id}","starred_url":"https://api.github.com/users/dspatil7768/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dspatil7768/subscriptions","organizations_url":"https://api.github.com/users/dspatil7768/orgs","repos_url":"https://api.github.com/users/dspatil7768/repos","events_url":"https://api.github.com/users/dspatil7768/events{/privacy}","received_events_url":"https://api.github.com/users/dspatil7768/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":{"login":"zet809","id":8222144,"node_id":"MDQ6VXNlcjgyMjIxNDQ=","avatar_url":"https://avatars2.githubusercontent.com/u/8222144?v=4","gravatar_id":"","url":"https://api.github.com/users/zet809","html_url":"https://github.com/zet809","followers_url":"https://api.github.com/users/zet809/followers","following_url":"https://api.github.com/users/zet809/following{/other_user}","gists_url":"https://api.github.com/users/zet809/gists{/gist_id}","starred_url":"https://api.github.com/users/zet809/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zet809/subscriptions","organizations_url":"https://api.github.com/users/zet809/orgs","repos_url":"https://api.github.com/users/zet809/repos","events_url":"https://api.github.com/users/zet809/events{/privacy}","received_events_url":"https://api.github.com/users/zet809/received_events","type":"User","site_admin":false},"assignees":[{"login":"zet809","id":8222144,"node_id":"MDQ6VXNlcjgyMjIxNDQ=","avatar_url":"https://avatars2.githubusercontent.com/u/8222144?v=4","gravatar_id":"","url":"https://api.github.com/users/zet809","html_url":"https://github.com/zet809","followers_url":"https://api.github.com/users/zet809/followers","following_url":"https://api.github.com/users/zet809/following{/other_user}","gists_url":"https://api.github.com/users/zet809/gists{/gist_id}","starred_url":"https://api.github.com/users/zet809/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zet809/subscriptions","organizations_url":"https://api.github.com/users/zet809/orgs","repos_url":"https://api.github.com/users/zet809/repos","events_url":"https://api.github.com/users/zet809/events{/privacy}","received_events_url":"https://api.github.com/users/zet809/received_events","type":"User","site_admin":false}],"milestone":null,"comments":4,"created_at":"2018-08-13T18:46:38Z","updated_at":"2018-09-03T05:28:35Z","closed_at":null,"author_association":"NONE","body":"@zet809 Trying installing latest `xCAT 2.14.2` on Centos 7.5 with OpenHPC but facing an issue after booting node i.e compute node dont get OPA ip and `ib0` interface down.\r\n\r\nBelow is node definition:\r\n```\r\n[root@shrestha ~]# lsdef gpu02\r\nObject name: gpu02\r\n    arch=x86_64\r\n    bmc=172.40.1.169\r\n    bmcpassword=calvin\r\n    bmcusername=root\r\n    chain=runcmd=bmcsetup,osimage=centos7.5-x86_64-netboot-cuda\r\n    cons=ipmi\r\n    currstate=netboot centos7.5-x86_64-compute\r\n    groups=all,gpu\r\n    ip=172.10.1.169\r\n    mac=20:04:0F:E6:EC:C8\r\n    mgt=ipmi\r\n    netboot=xnba\r\n    nicips.ib0=172.30.1.169\r\n    nicnetworks.ib0=ib0\r\n    nictypes.ib0=Omnipath\r\n    os=centos7.5\r\n    postbootscripts=otherpkgs,confignics\r\n    postscripts=syslog,remoteshell,syncfiles\r\n    profile=compute\r\n    provmethod=centos7.5-x86_64-netboot-cuda\r\n    serialport=0\r\n    serialspeed=115200\r\n    status=booted\r\n    statustime=08-13-2018 23:48:10\r\n    updatestatus=synced\r\n    updatestatustime=08-13-2018 23:54:25\r\n```\r\nBelow is some data \r\n\r\n```\r\n[root@shrestha ~]# ansible gpu02 -m shell -a \"ip a\"\r\ngpu02 | SUCCESS | rc=0 >>\r\n1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000\r\n    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00\r\n    inet 127.0.0.1/8 scope host lo\r\n       valid_lft forever preferred_lft forever\r\n    inet6 ::1/128 scope host\r\n       valid_lft forever preferred_lft forever\r\n2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000\r\n    link/ether 20:04:0f:e6:ec:c8 brd ff:ff:ff:ff:ff:ff\r\n    inet 172.10.1.169/24 brd 172.10.1.255 scope global dynamic eno1\r\n       valid_lft 42829sec preferred_lft 42829sec\r\n    inet6 fe80::2204:fff:fee6:ecc8/64 scope link\r\n       valid_lft forever preferred_lft forever\r\n3: eno2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000\r\n    link/ether 20:04:0f:e6:ec:c9 brd ff:ff:ff:ff:ff:ff\r\n4: eno3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000\r\n    link/ether 20:04:0f:e6:ec:ca brd ff:ff:ff:ff:ff:ff\r\n5: eno4: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000\r\n    link/ether 20:04:0f:e6:ec:cb brd ff:ff:ff:ff:ff:ff\r\n6: ib0: <BROADCAST,MULTICAST> mtu 4092 qdisc noop state DOWN group default qlen 256\r\n    link/infiniband 80:00:00:02:fe:80:00:00:00:00:00:00:00:11:75:01:01:08:82:c9 brd 00:ff:ff:ff:ff:12:40:1b:80:01:00:00:00:00:00:00:ff:ff:ff:ff\r\n```\r\n`lscpi` output\r\n```\r\n[root@shrestha ~]# ansible gpu02 -m shell -a \"lspci -vvv |grep HFI\"\r\ngpu02 | SUCCESS | rc=0 >>\r\naf:00.0 Fabric controller: Intel Corporation Omni-Path HFI Silicon 100 Series [discrete] (rev 11)\r\n        Subsystem: Intel Corporation Omni-Path HFI Silicon 100 Series [discrete]\r\n```\r\n```\r\n[root@shrestha ~]# tabdump networks\r\n#netname,net,mask,mgtifname,gateway,dhcpserver,tftpserver,nameservers,ntpservers,logservers,dynamicrange,staticrange,staticrangeincrement,nodehostname,ddnsdomain,vlanid,domain,mtu,comments,disable\r\n\"10_208_34_0-255_255_254_0\",\"10.208.34.0\",\"255.255.254.0\",\"em1\",\"<xcatmaster>\",,\"<xcatmaster>\",,,,,,,,,,,\"1500\",,\r\n\"172_10_1_0-255_255_255_0\",\"172.10.1.0\",\"255.255.255.0\",\"em2\",\"172.10.1.150\",,\"<xcatmaster>\",,,,,,,,,,,\"1500\",,\r\n\"192_168_122_0-255_255_255_0\",\"192.168.122.0\",\"255.255.255.0\",\"virbr0\",\"<xcatmaster>\",,\"<xcatmaster>\",,,,,,,,,,,\"1500\",,\r\n\"172_40_1_0-255_255_255_0\",\"172.40.1.0\",\"255.255.255.0\",\"em3\",\"<xcatmaster>\",,\"<xcatmaster>\",,,,,,,,,,,\"1500\",,\r\n\"172_30_1_0-255_255_255_0\",\"172.30.1.0\",\"255.255.255.0\",\"ib0\",\"<xcatmaster>\",,\"<xcatmaster>\",,,,,,,,,,,\"2044\",,\r\n```\r\n\r\nRefering below document \r\n\r\n[Install_guide-CentOS7-xCAT-Stateless-SLURM-1.3.5-x86_64.pdf](https://github.com/xcat2/xcat-core/files/2284305/Install_guide-CentOS7-xCAT-Stateless-SLURM-1.3.5-x86_64.pdf)\r\n"},"comment":{"url":"https://api.github.com/repos/xcat2/xcat-core/issues/comments/418003974","html_url":"https://github.com/xcat2/xcat-core/issues/5506#issuecomment-418003974","issue_url":"https://api.github.com/repos/xcat2/xcat-core/issues/5506","id":418003974,"node_id":"MDEyOklzc3VlQ29tbWVudDQxODAwMzk3NA==","user":{"login":"dspatil7768","id":22992849,"node_id":"MDQ6VXNlcjIyOTkyODQ5","avatar_url":"https://avatars2.githubusercontent.com/u/22992849?v=4","gravatar_id":"","url":"https://api.github.com/users/dspatil7768","html_url":"https://github.com/dspatil7768","followers_url":"https://api.github.com/users/dspatil7768/followers","following_url":"https://api.github.com/users/dspatil7768/following{/other_user}","gists_url":"https://api.github.com/users/dspatil7768/gists{/gist_id}","starred_url":"https://api.github.com/users/dspatil7768/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dspatil7768/subscriptions","organizations_url":"https://api.github.com/users/dspatil7768/orgs","repos_url":"https://api.github.com/users/dspatil7768/repos","events_url":"https://api.github.com/users/dspatil7768/events{/privacy}","received_events_url":"https://api.github.com/users/dspatil7768/received_events","type":"User","site_admin":false},"created_at":"2018-09-03T05:28:35Z","updated_at":"2018-09-03T05:28:35Z","author_association":"NONE","body":"@robin2008 as told, got by changing `nictypes.ib0=Omnipath` to `nictypes.ib0=Infiniband`\r\n\r\n```\r\n[root@shrestha1 ~]# lsdef gpu02\r\nObject name: gpu02\r\n    arch=x86_64\r\n    bmc=172.40.1.169\r\n    bmcpassword=calvin\r\n    bmcusername=root\r\n    chain=runcmd=bmcsetup,osimage=centos7.4-x86_64-netboot-cuda\r\n    cons=ipmi\r\n    consoleenabled=1\r\n    currstate=netboot centos7.4-x86_64-compute\r\n    groups=all,gpu\r\n    ip=172.10.1.169\r\n    mac=20:04:0F:E6:EC:C8\r\n    mgt=ipmi\r\n    netboot=xnba\r\n    nicips.ib0=172.30.1.169\r\n    nicnetworks.ib0=ib0\r\n    nictypes.ib0=Infiniband\r\n    os=centos7.4\r\n    postbootscripts=otherpkgs\r\n    postscripts=syslog,remoteshell,syncfiles,confignetwork -s,confignics,lustre.sh\r\n    profile=compute\r\n    provmethod=centos7.4-x86_64-netboot-compute\r\n    serialport=0\r\n    serialspeed=115200\r\n    status=booted\r\n    statustime=09-03-2018 10:51:02\r\n    updatestatus=synced\r\n    updatestatustime=08-31-2018 18:48:15\r\n\r\n```\r\n\r\n```\r\n[root@gpu02 ~]# cat /var/log/xcat/xcat.log\r\nMon Sep  3 10:51:48 IST 2018 [info]: xcat.xcatdsklspost: trying to download postscripts...\r\nMon Sep  3 10:51:48 IST 2018 [info]: xcat.xcatdsklspost: postscripts downloaded successfully\r\nMon Sep  3 10:51:48 IST 2018 [info]: xcat.xcatdsklspost: trying to get mypostscript from 172.10.1.149...\r\nMon Sep  3 10:51:48 IST 2018 [info]: xcat.mypostscript: Running postbootscript: syslog\r\ngrep: /etc/rsyslog.d/remote.conf: No such file or directory\r\ngrep: /etc/rsyslog.d/remote.conf: No such file or directory\r\nMon Sep  3 10:51:48 IST 2018 [info]: xcat.mypostscript: postbootscript syslog return with 0\r\nMon Sep  3 10:51:48 IST 2018 [info]: xcat.mypostscript: Running postbootscript: remoteshell\r\n\r\nMon Sep  3 10:51:50 IST 2018 [info]: xcat.mypostscript: postbootscript remoteshell return with 0\r\nMon Sep  3 10:51:50 IST 2018 [info]: xcat.mypostscript: Running postbootscript: syncfiles\r\nMon Sep  3 10:51:52 IST 2018 [info]: xcat.mypostscript: postbootscript syncfiles return with 0\r\nMon Sep  3 10:51:52 IST 2018 [info]: xcat.mypostscript: Running postbootscript: confignetwork\r\n++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++\r\n[I]: configure the install nic eno1.\r\n[I]: configeth on gpu02: os type: redhat\r\n['/etc/sysconfig/network-scripts/ifcfg-eno1']\r\n[I]: >> DEVICE=eno1\r\n[I]: >> IPADDR=172.10.1.169\r\n[I]: >> NETMASK=255.255.255.0\r\n[I]: >> BOOTPROTO=static\r\n[I]: >> ONBOOT=yes\r\n[I]: >> HWADDR=20:04:0F:E6:EC:C8\r\n[I]: >> MTU=1500\r\n++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++\r\n[I]: All valid nics and device list:\r\n[I]: ib0\r\n[I]: NetworkManager is inactive.\r\n++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++\r\nconfigure nic and its device : ib0\r\n[I]: Call configib for IB nics: ib0, ports:\r\n[I]: NIC_IBNICS=ib0 NIC_IBAPORTS= configib\r\nMon Sep  3 10:52:01 IST 2018 [info]: xcat.mypostscript: postbootscript confignetwork return with 0\r\nMon Sep  3 10:52:01 IST 2018 [info]: xcat.mypostscript: Running postbootscript: confignics\r\nconfignics on gpu02: config install nic:0, remove: 0, iba ports:\r\nib0!172.30.1.169\r\nconfignics on gpu02: executed script: configib for nics: ib0, ports:\r\nMon Sep  3 10:52:07 IST 2018 [info]: xcat.mypostscript: postbootscript confignics return with 0\r\nMon Sep  3 10:52:07 IST 2018 [info]: xcat.mypostscript: Running postbootscript: lustre.sh\r\nLNET configured\r\nMon Sep  3 10:52:08 IST 2018 [info]: xcat.mypostscript: postbootscript lustre.sh return with 0\r\nMon Sep  3 10:52:08 IST 2018 [info]: xcat.mypostscript: Running postbootscript: otherpkgs\r\n  Did not install any extra packages.\r\nMon Sep  3 10:52:08 IST 2018 [info]: xcat.mypostscript: postbootscript otherpkgs return with 0\r\nMon Sep  3 10:52:08 IST 2018 [debug]: xcat.mypostscript: node booted successfully, reporting status...\r\nMon Sep  3 10:52:08 IST 2018 [info]: xcat.mypostscript: provision completed.(gpu02)\r\n\r\n```"}}
{"actor":{"display_login":"dspatil7768"}}