{"action":"created","issue":{"url":"https://api.github.com/repos/ceph/ceph-ansible/issues/496","labels_url":"https://api.github.com/repos/ceph/ceph-ansible/issues/496/labels{/name}","comments_url":"https://api.github.com/repos/ceph/ceph-ansible/issues/496/comments","events_url":"https://api.github.com/repos/ceph/ceph-ansible/issues/496/events","html_url":"https://github.com/ceph/ceph-ansible/issues/496","id":129275071,"number":496,"title":"owner for osds and journal device partition are rnot set correctly with infernalis","user":{"login":"mistur","id":15873680,"avatar_url":"https://avatars.githubusercontent.com/u/15873680?v=3","gravatar_id":"","url":"https://api.github.com/users/mistur","html_url":"https://github.com/mistur","followers_url":"https://api.github.com/users/mistur/followers","following_url":"https://api.github.com/users/mistur/following{/other_user}","gists_url":"https://api.github.com/users/mistur/gists{/gist_id}","starred_url":"https://api.github.com/users/mistur/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mistur/subscriptions","organizations_url":"https://api.github.com/users/mistur/orgs","repos_url":"https://api.github.com/users/mistur/repos","events_url":"https://api.github.com/users/mistur/events{/privacy}","received_events_url":"https://api.github.com/users/mistur/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":4,"created_at":"2016-01-27T21:30:25Z","updated_at":"2016-01-28T12:51:08Z","closed_at":null,"body":"OS : Ubuntu 14.04\r\nceph version : infernalis\r\n\r\nJournals are on ssd raw partition : \r\n\r\ngroup_vars/osds :\r\n\r\n`crush_location: false`\r\n`osd_crush_location: \"'root={{ ceph_crush_root }} rack={{ ceph_crush_rack }} host={{ ansible_hostname }}'\"`\r\n``\r\n`devices:`\r\n`  - /dev/sdb`\r\n`  - /dev/sdc`\r\n``\r\n`raw_multi_journal: true`\r\n`raw_journal_devices:`\r\n`  - /dev/sdl1`\r\n`  - /dev/sdl2`\r\n\r\nI must create SSD partitons before to run the playbook, \r\n\r\n`brw-rw---- 1 root disk 8, 177 jan 27 22:00 /dev/sdl1`\r\n`brw-rw---- 1 root disk 8, 178 jan 27 22:00 /dev/sdl2`\r\n\r\nsince infernalis, OSD runs with the user ceph so both osd disks and journal partitions seem to be owned by the ceph user.\r\n\r\nto set the owner of the disk I run that's script before \r\n\r\n`for disk in sdb sdc  ; do `\r\n`\tif ! grep -q ${disk} /etc/udev/rules.d/89-ceph-journal.rules ; then`\r\n`\t\techo \"KERNEL==\\\"${disk}?\\\" SUBSYSTEM==\\\"block\\\" OWNER=\\\"ceph\\\" GROUP=\\\"disk\\\" MODE=\\\"0660\\\"\" >> /etc/udev/rules.d/89-ceph-journal.rules`\r\n`\tfi`\r\n`done`\r\n\r\nso when the ceph-disk create OSDs partitions, udev set the good owner.\r\n \r\nmaybe ceph-ansible should fill /etc/udev/rules.d/89-ceph-journal.rules ?\r\n\r\nand for the journal partition, it's not possible to change the owner of the partition before because the user ceph is not create yet, and the playbook does not change the owner of journal partition. so OSDs fails to start, we must change the user of journal partitions and run the playbook again to starts OSDSs\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/ceph/ceph-ansible/issues/comments/176166056","html_url":"https://github.com/ceph/ceph-ansible/issues/496#issuecomment-176166056","issue_url":"https://api.github.com/repos/ceph/ceph-ansible/issues/496","id":176166056,"user":{"login":"leseb","id":912735,"avatar_url":"https://avatars.githubusercontent.com/u/912735?v=3","gravatar_id":"","url":"https://api.github.com/users/leseb","html_url":"https://github.com/leseb","followers_url":"https://api.github.com/users/leseb/followers","following_url":"https://api.github.com/users/leseb/following{/other_user}","gists_url":"https://api.github.com/users/leseb/gists{/gist_id}","starred_url":"https://api.github.com/users/leseb/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/leseb/subscriptions","organizations_url":"https://api.github.com/users/leseb/orgs","repos_url":"https://api.github.com/users/leseb/repos","events_url":"https://api.github.com/users/leseb/events{/privacy}","received_events_url":"https://api.github.com/users/leseb/received_events","type":"User","site_admin":false},"created_at":"2016-01-28T12:51:08Z","updated_at":"2016-01-28T12:51:08Z","body":"Did you try to simply set:\r\n\r\n```\r\nraw_journal_devices:\r\n- /dev/sdl\r\n- /dev/sdl\r\n```\r\n\r\nAnd let ceph-disk configure the device.\r\nOne a test machine I have one OSD and one dedicated journal and I have the following permissions:\r\n\r\n```\r\nroot@ceph-ansible:~# stat /dev/vdc\r\n  File: ‘/dev/vdc’\r\n  Size: 0               Blocks: 0          IO Block: 4096   block special file\r\nDevice: 5h/5d   Inode: 1244        Links: 1     Device type: fd,20\r\nAccess: (0660/brw-rw----)  Uid: (    0/    root)   Gid: (    6/    disk)\r\nAccess: 2016-01-27 23:47:59.685809423 +0000\r\nModify: 2016-01-27 23:47:37.597748007 +0000\r\nChange: 2016-01-27 23:47:37.597748007 +0000\r\n Birth: -\r\nroot@ceph-ansible:~# stat /dev/vdc1\r\n  File: ‘/dev/vdc1’\r\n  Size: 0               Blocks: 0          IO Block: 4096   block special file\r\nDevice: 5h/5d   Inode: 111559      Links: 1     Device type: fd,21\r\nAccess: (0660/brw-rw----)  Uid: (64045/    ceph)   Gid: (64045/    ceph)\r\nAccess: 2016-01-27 23:48:01.073813177 +0000\r\nModify: 2016-01-27 23:48:10.377838026 +0000\r\nChange: 2016-01-27 23:48:10.377838026 +0000\r\n Birth: -\r\nroot@ceph-ansible:~# stat /dev/vdb\r\n  File: ‘/dev/vdb’\r\n  Size: 0               Blocks: 0          IO Block: 4096   block special file\r\nDevice: 5h/5d   Inode: 1243        Links: 1     Device type: fd,10\r\nAccess: (0660/brw-rw----)  Uid: (    0/    root)   Gid: (    6/    disk)\r\nAccess: 2016-01-27 23:48:00.649812031 +0000\r\nModify: 2016-01-27 23:48:00.649812031 +0000\r\nChange: 2016-01-27 23:48:00.649812031 +0000\r\n Birth: -\r\nroot@ceph-ansible:~# stat /dev/vdb1\r\n  File: ‘/dev/vdb1’\r\n  Size: 0               Blocks: 0          IO Block: 4096   block special file\r\nDevice: 5h/5d   Inode: 112750      Links: 1     Device type: fd,11\r\nAccess: (0660/brw-rw----)  Uid: (64045/    ceph)   Gid: (64045/    ceph)\r\nAccess: 2016-01-27 23:47:59.845809856 +0000\r\nModify: 2016-01-27 23:47:58.037804951 +0000\r\nChange: 2016-01-27 23:47:58.037804951 +0000\r\n Birth: -\r\n```\r\n\r\nHere `vdb` is the OSD and `vdc` is the journal."}}