[ceph-users] Filestore to Bluestore migration question

Hayashida, Mami mami.hayashida at uky.edu
Tue Nov 6 07:30:05 PST 2018


So, currently this is what /var/lib/ceph/osd/ceph-60 shows.  Is it not
correct?   I don't know what I should expect to see.

root at osd1:~# ls -l /var/lib/ceph/osd/ceph-60
total 86252
-rw-r--r-- 1 ceph ceph         384 Nov  2 16:20 activate.monmap
-rw-r--r-- 1 ceph ceph 10737418240 Nov  5 16:32 block
lrwxrwxrwx 1 ceph ceph          14 Nov  2 16:20 block.db -> /dev/ssd0/db60
-rw-r--r-- 1 ceph ceph           2 Nov  2 16:20 bluefs
-rw-r--r-- 1 ceph ceph          37 Nov  2 16:20 ceph_fsid
-rw-r--r-- 1 ceph ceph          37 Nov  2 16:20 fsid
-rw------- 1 ceph ceph          57 Nov  2 16:20 keyring
-rw-r--r-- 1 ceph ceph           8 Nov  2 16:20 kv_backend
-rw-r--r-- 1 ceph ceph          21 Nov  2 16:20 magic
-rw-r--r-- 1 ceph ceph           4 Nov  2 16:20 mkfs_done
-rw-r--r-- 1 ceph ceph          41 Nov  2 16:20 osd_key
-rw-r--r-- 1 ceph ceph           6 Nov  2 16:20 ready
-rw-r--r-- 1 ceph ceph          10 Nov  2 16:20 type
-rw-r--r-- 1 ceph ceph           3 Nov  2 16:20 whoami

The disk I am using for this osd (osd.60) is a 3.7TB hdd.

lsblk shows

sdh              8:112  0   3.7T  0 disk
└─hdd60-data60 252:1    0   3.7T  0 lvm

and "ceph osd tree" shows
60   hdd    3.63689         osd.60         up  1.00000 1.00000



On Mon, Nov 5, 2018 at 11:23 PM, Hector Martin <hector at marcansoft.com>
wrote:

> On 11/6/18 6:03 AM, Hayashida, Mami wrote:
> > WOW.  With you two guiding me through every step, the 10 OSDs in
> > question are now added back to the cluster as Bluestore disks!!!  Here
> > are my responses to the last email from Hector:
> >
> > 1. I first checked the permissions and they looked like this
> >
> > root at osd1:/var/lib/ceph/osd/ceph-60# ls -l
> > total 56
> > -rw-r--r-- 1 ceph ceph         384 Nov  2 16:20 activate.monmap
> > -rw-r--r-- 1 ceph ceph 10737418240 Nov  2 16:20 block
> > lrwxrwxrwx 1 ceph ceph          14 Nov  2 16:20 block.db ->
> /dev/ssd0/db60
>
> Wait.
>
> That's not right, is it? That's a 10GB raw file being used as a
> BlueStore block device. It should be a symlink to an LVM volume, not a
> file.
>
> Can you check the `block` files/symlinks again for all the OSDs and also
> what is mounted on the OSD directories? It should be tmpfs directories
> with symlinks to block devices. I'm not sure what happened there.
>
> --
> Hector Martin (hector at marcansoft.com)
> Public Key: https://mrcn.st/pub
>



-- 
*Mami Hayashida*

*Research Computing Associate*
Research Computing Infrastructure
University of Kentucky Information Technology Services
301 Rose Street | 102 James F. Hardymon Building
Lexington, KY 40506-0495
mami.hayashida at uky.edu
(859)323-7521
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ceph.com/pipermail/ceph-users-ceph.com/attachments/20181106/4c2ffa73/attachment.html>


More information about the ceph-users mailing list