{"action":"closed","issue":{"url":"https://api.github.com/repos/OpenNebula/addon-linstor/issues/2","repository_url":"https://api.github.com/repos/OpenNebula/addon-linstor","labels_url":"https://api.github.com/repos/OpenNebula/addon-linstor/issues/2/labels{/name}","comments_url":"https://api.github.com/repos/OpenNebula/addon-linstor/issues/2/comments","events_url":"https://api.github.com/repos/OpenNebula/addon-linstor/issues/2/events","html_url":"https://github.com/OpenNebula/addon-linstor/issues/2","id":385301711,"node_id":"MDU6SXNzdWUzODUzMDE3MTE=","number":2,"title":"After (hard) reboot of storage node state of (disk-)resource (attached to a VM) is unknown on rebooted node","user":{"login":"ulide4","id":38238648,"node_id":"MDQ6VXNlcjM4MjM4NjQ4","avatar_url":"https://avatars1.githubusercontent.com/u/38238648?v=4","gravatar_id":"","url":"https://api.github.com/users/ulide4","html_url":"https://github.com/ulide4","followers_url":"https://api.github.com/users/ulide4/followers","following_url":"https://api.github.com/users/ulide4/following{/other_user}","gists_url":"https://api.github.com/users/ulide4/gists{/gist_id}","starred_url":"https://api.github.com/users/ulide4/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ulide4/subscriptions","organizations_url":"https://api.github.com/users/ulide4/orgs","repos_url":"https://api.github.com/users/ulide4/repos","events_url":"https://api.github.com/users/ulide4/events{/privacy}","received_events_url":"https://api.github.com/users/ulide4/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":3,"created_at":"2018-11-28T14:46:55Z","updated_at":"2018-11-29T13:43:55Z","closed_at":"2018-11-29T13:43:55Z","author_association":"NONE","body":"Hi again,\r\nI am currently in the status of testing what can go wrong and have switched off the passive storage node (resource state=unused) and switched it on again to see if replication starts again.\r\nSee detailed history below (FYI, that the text is partly crossed  is unintended)\r\n\r\nSETUP: \r\nsrv485 -> linstor controller & OpenNebula Frontend\r\nsrv484 & srv483 -> linstor satellite & storage node & OpenNebula KVM host, \r\ndatatore policy -> LINSTOR_AUTO_PLACE = 2\r\nOS: Centos 7.5 (same error behaviour on Ubuntu 18.04 btw)\r\n\r\nVM creation, live migration, VM deletion is working fine. \r\nIs there conceptually something I am missing or is this unexpected behavior? \r\n\r\noneadmin@srv485:~$ linstor resource list\r\n╭────────────────────────────────────────────────────────╮\r\n┊ ResourceName       ┊ Node   ┊ Port ┊ Usage  ┊    State ┊\r\n╞┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄╡\r\n┊ OpenNebula-Image-0 ┊ srv483 ┊ 7000 ┊ Unused ┊ UpToDate ┊\r\n┊ OpenNebula-Image-0 ┊ srv484 ┊ 7000 ┊ Unused ┊ UpToDate ┊\r\n╰────────────────────────────────────────────────────────╯\r\n\r\noneadmin@srv485:~$ onetemplate instantiate 0\r\nVM ID: 0\r\n\r\noneadmin@srv485:~$ linstor resource list\r\n╭──────────────────────────────────────────────────────────────────╮\r\n┊ ResourceName                 ┊ Node   ┊ Port ┊ Usage  ┊    State ┊\r\n╞┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄╡\r\n┊ OpenNebula-Image-0           ┊ srv483 ┊ 7000 ┊ Unused ┊ UpToDate ┊\r\n┊ OpenNebula-Image-0           ┊ srv484 ┊ 7000 ┊ Unused ┊ UpToDate ┊\r\n┊ OpenNebula-Image-0-vm0-disk0 ┊ srv483 ┊ 7001 ┊ Unused ┊ UpToDate ┊\r\n┊ OpenNebula-Image-0-vm0-disk0 ┊ srv484 ┊ 7001 ┊ InUse  ┊ UpToDate ┊\r\n╰──────────────────────────────────────────────────────────────────╯\r\n\r\nSWITCHING OFF NODE srv483 \r\n\r\n\r\noneadmin@srv485:~$ linstor resource list\r\n╭──────────────────────────────────────────────────────────────────╮\r\n┊ ResourceName                 ┊ Node   ┊ Port ┊ Usage  ┊    State ┊\r\n╞┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄╡\r\n┊ OpenNebula-Image-0           ┊ srv483 ┊ 7000 ┊        ┊  Unknown ┊\r\n┊ OpenNebula-Image-0           ┊ srv484 ┊ 7000 ┊ Unused ┊ UpToDate ┊\r\n┊ OpenNebula-Image-0-vm0-disk0 ┊ srv483 ┊ 7001 ┊        ┊  Unknown ┊\r\n┊ OpenNebula-Image-0-vm0-disk0 ┊ srv484 ┊ 7001 ┊ InUse  ┊ UpToDate ┊\r\n╰──────────────────────────────────────────────────────────────────╯\r\n\r\noneadmin@srv485:~$ linstor node list\r\n╭─────────────────────────────────────────────────────────────────────╮\r\n┊ Node   ┊ NodeType  ┊ Addresses                            ┊ State   ┊\r\n╞┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄╡\r\n┊ srv483 ┊ SATELLITE ┊ 100.80.0.15,100.80.0.15:3366 (PLAIN) ┊ OFFLINE ┊\r\n┊ srv484 ┊ SATELLITE ┊ 100.80.0.16,100.80.0.16:3366 (PLAIN) ┊ Online  ┊\r\n╰─────────────────────────────────────────────────────────────────────╯\r\n\r\nSWITCHING ON node srv483\r\n\r\noneadmin@srv485:~$  linstor node list\r\n╭────────────────────────────────────────────────────────────────────╮\r\n┊ Node   ┊ NodeType  ┊ Addresses                            ┊ State  ┊\r\n╞┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄╡\r\n┊ srv483 ┊ SATELLITE ┊ 100.80.0.15,100.80.0.15:3366 (PLAIN) ┊ Online ┊\r\n┊ srv484 ┊ SATELLITE ┊ 100.80.0.16,100.80.0.16:3366 (PLAIN) ┊ Online ┊\r\n╰────────────────────────────────────────────────────────────────────╯\r\noneadmin@srv485:~$ linstor resource list\r\n╭──────────────────────────────────────────────────────────────────╮\r\n┊ ResourceName                 ┊ Node   ┊ Port ┊ Usage  ┊    State ┊\r\n╞┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄╡\r\n┊ OpenNebula-Image-0           ┊ srv483 ┊ 7000 ┊ Unused ┊ UpToDate ┊\r\n┊ OpenNebula-Image-0           ┊ srv484 ┊ 7000 ┊ Unused ┊ UpToDate ┊\r\n┊ OpenNebula-Image-0-vm0-disk0 ┊ srv483 ┊ 7001 ┊        ┊  Unknown ┊\r\n┊ OpenNebula-Image-0-vm0-disk0 ┊ srv484 ┊ 7001 ┊ InUse  ┊ UpToDate ┊\r\n╰──────────────────────────────────────────────────────────────────╯\r\n\r\noneadmin@srv485:~$ linstor error-reports list\r\n╭────────────────────────────────────────────────────────────╮\r\n┊ Nr. ┊ Id                    ┊ Datetime            ┊ Node   ┊\r\n╞┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄╡\r\n┊   1 ┊ 5BFE9E8A-90340-000000 ┊ 2018-11-28 14:56:47 ┊ srv483 ┊\r\n╰────────────────────────────────────────────────────────────╯\r\noneadmin@srv485:~$ linstor error-reports show  5BFE9E8A-90340-000000\r\nERROR REPORT 5BFE9E8A-90340-000000\r\n\r\n============================================================\r\n\r\nApplication:                        LINBIT® LINSTOR\r\nModule:                             Satellite\r\nVersion:                            0.7.3\r\nBuild ID:                           6e47dd2cb440e65727889f4c2f7423996153bdb3\r\nBuild time:                         2018-11-22T10:55:51+00:00\r\nError time:                         2018-11-28 14:56:47\r\nNode:                               srv483\r\n\r\n============================================================\r\n\r\nReported error:\r\n===============\r\n\r\nDescription:\r\n    Operations on resource 'OpenNebula-Image-0-vm0-disk0' were aborted\r\nCause:\r\n    Meta data creation for volume 0 failed because the execution of an external command failed\r\nCorrection:\r\n    - Check whether the required software is installed\r\n    - Check whether the application's search path includes the location\r\n      of the external software\r\n    - Check whether the application has execute permission for the external command\r\n\r\nCategory:                           LinStorException\r\nClass name:                         ResourceException\r\nClass canonical name:               com.linbit.linstor.core.DrbdDeviceHandler.ResourceException\r\nGenerated at:                       Method 'createResourceMetaData', Source file 'DrbdDeviceHandler.java', Line #1524\r\n\r\nError message:                      Meta data creation for resource 'OpenNebula-Image-0-vm0-disk0' volume 0 failed\r\n\r\nError context:\r\n    Meta data creation for resource 'OpenNebula-Image-0-vm0-disk0' volume 0 failed\r\n\r\nCall backtrace:\r\n\r\n    Method                                   Native Class:Line number\r\n    createResourceMetaData                   N      com.linbit.linstor.core.DrbdDeviceHandler:1524\r\n    createResource                           N      com.linbit.linstor.core.DrbdDeviceHandler:1138\r\n    dispatchResource                         N      com.linbit.linstor.core.DrbdDeviceHandler:364\r\n    run                                      N      com.linbit.linstor.core.DeviceManagerImpl$DeviceHandlerInvocation:1225\r\n    run                                      N      com.linbit.WorkerPool$WorkerThread:179\r\n\r\nCaused by:\r\n==========\r\n\r\nDescription:\r\n    Execution of the external command 'drbdadm' failed.\r\nCause:\r\n    The external command exited with error code 20.\r\nCorrection:\r\n    - Check whether the external program is operating properly.\r\n    - Check whether the command line is correct.\r\n      Contact a system administrator or a developer if the command line is no longer valid\r\n      for the installed version of the external program.\r\nAdditional information:\r\n    The full command line executed was:\r\n    drbdadm -vvv --max-peers 7 -- --force create-md OpenNebula-Image-0-vm0-disk0/0\r\n\r\n    The external command sent the following output data:\r\n    drbdmeta 1001 v09 /dev/vg-drbdpool/OpenNebula-Image-0-vm0-disk0_00000 internal create-md 7 --force \r\n\r\n\r\n    The external command sent the follwing error information:\r\n    open(/dev/vg-drbdpool/OpenNebula-Image-0-vm0-disk0_00000) failed: No such file or directory\r\n    open(/dev/vg-drbdpool/OpenNebula-Image-0-vm0-disk0_00000) failed: No such file or directory\r\n    Command 'drbdmeta 1001 v09 /dev/vg-drbdpool/OpenNebula-Image-0-vm0-disk0_00000 internal create-md 7 --force' terminated with exit code 20\r\n\r\n\r\nCategory:                           LinStorException\r\nClass name:                         ExtCmdFailedException\r\nClass canonical name:               com.linbit.extproc.ExtCmdFailedException\r\nGenerated at:                       Method 'execute', Source file 'DrbdAdm.java', Line #437\r\n\r\nError message:                      The external command 'drbdadm' exited with error code 20\r\n\r\n\r\nCall backtrace:\r\n\r\n    Method                                   Native Class:Line number\r\n    execute                                  N      com.linbit.drbd.DrbdAdm:437\r\n    simpleAdmCommand                         N      com.linbit.drbd.DrbdAdm:388\r\n    createMd                                 N      com.linbit.drbd.DrbdAdm:217\r\n    createVolumeMetaData                     N      com.linbit.linstor.core.DrbdDeviceHandler:1060\r\n    createResourceMetaData                   N      com.linbit.linstor.core.DrbdDeviceHandler:1489\r\n    createResource                           N      com.linbit.linstor.core.DrbdDeviceHandler:1138\r\n    dispatchResource                         N      com.linbit.linstor.core.DrbdDeviceHandler:364\r\n    run                                      N      com.linbit.linstor.core.DeviceManagerImpl$DeviceHandlerInvocation:1225\r\n    run                                      N      com.linbit.WorkerPool$WorkerThread:179\r\n\r\n\r\nEND OF ERROR REPORT.\r\n\r\n\r\n"}}
{"actor":{"display_login":"ulide4"}}