[ceph-users] Resharding issues / How long does it take?

Martin Emrich martin.emrich at empolis.com
Thu Dec 7 07:05:06 PST 2017


Hi all!

Apparently, one of my buckets went wonko during automatic resharding, the frontend application only gehts a timeout after 90s.
After an attempt to fix the index using “radosgw-admin bucket check –fix”, I tried to reshard id (6,3GB of data in ca. 230000 objects).

The resharding command is now running for over an hour. No significant load on any of the 18 OSDs, the host running radosgw-admin or on one of the three radosgw hosts. The OSDs are beefy machines with HDDs for Data and SSDs for index pools. Running 12.2.2. 
How long should the resharding take? For a few minutes, radosgw-admin seems quite busy, but now it seems to only sit there at only a few % of CPU usage.

“radosgw-admin reshard list” reports an empty list. Reshard status reports

[
    {
        "reshard_status": 1,
        "new_bucket_instance_id": "c2ffcb0f-a9a3-4360-a9be-5edef965449a.6860125.1",
        "num_shards": 10
    }
]

I have a feeling that the bucket index is still damaged/incomplete/inconsistent. What does the message

*** NOTICE: operation will not remove old bucket index objects ***
***         these will need to be removed manually             ***

mean? How can I clean up manually?

Thanks,

Martin
 



More information about the ceph-users mailing list