[ceph-users] Filestore to Bluestore migration question

Hayashida, Mami mami.hayashida at uky.edu
Tue Nov 6 13:12:43 PST 2018

This is becoming even more confusing. I got rid of those
ceph-disk at 6[0-9].service
(which had been symlinked to /dev/null).  Moved
/var/lib/ceph/osd/ceph-6[0-9] to  /var/...../osd_old/.  Then, I ran
`ceph-volume lvm activate --all`.  I got once again

root at osd1:~# ceph-volume lvm activate --all
--> Activating OSD ID 67 FSID 17cd6755-76f9-4160-906c-1bf13d09fb3d
Running command: mount -t tmpfs tmpfs /var/lib/ceph/osd/ceph-67
--> Absolute path not found for executable: restorecon
--> Ensure $PATH environment variable contains common executable locations
Running command: ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev
/dev/hdd67/data67 --path /var/lib/ceph/osd/ceph-67
 stderr: failed to read label for /dev/hdd67/data67: (2) No such file or
-->  RuntimeError: command returned non-zero exit status: 1

But when I ran `df` and `mount` ceph-67 is the only one that exists. (and
in  /var/lib/ceph/osd/)

root at osd1:~# df -h | grep ceph-6
tmpfs           126G     0  126G   0% /var/lib/ceph/osd/ceph-67

root at osd1:~# mount | grep ceph-6
tmpfs on /var/lib/ceph/osd/ceph-67 type tmpfs (rw,relatime)

root at osd1:~# ls /var/lib/ceph/osd/ | grep ceph-6

But in I cannot restart any of these 10 daemons (`systemctl start ceph-osd at 6

I am wondering if I should zap these 10 osds and start over although at
this point I am afraid even zapping may not be a simple task....

On Tue, Nov 6, 2018 at 3:44 PM, Hector Martin <hector at marcansoft.com> wrote:

> On 11/7/18 5:27 AM, Hayashida, Mami wrote:
> > 1. Stopped osd.60-69:  no problem
> > 2. Skipped this and went to #3 to check first
> > 3. Here, `find /etc/systemd/system | grep ceph-volume` returned
> > nothing.  I see in that directory
> >
> > /etc/systemd/system/ceph-disk at 60.service    # and 61 - 69.
> >
> > No ceph-volume entries.
> Get rid of those, they also shouldn't be there. Then `systemctl
> daemon-reload` and continue, see if you get into a good state. basically
> feel free to nuke anything in there related to OSD 60-69, since whatever
> is needed should be taken care of by the ceph-volume activation.
> --
> Hector Martin (hector at marcansoft.com)
> Public Key: https://mrcn.st/pub

*Mami Hayashida*

*Research Computing Associate*
Research Computing Infrastructure
University of Kentucky Information Technology Services
301 Rose Street | 102 James F. Hardymon Building
Lexington, KY 40506-0495
mami.hayashida at uky.edu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ceph.com/pipermail/ceph-users-ceph.com/attachments/20181106/b7ade4be/attachment.html>

More information about the ceph-users mailing list