[ceph-users] inexplicably slow bucket listing at top level

J. Eric Ivancich ivancich at redhat.com
Mon Nov 5 12:10:41 PST 2018


I did make an inquiry and someone here does have some experience w/ the
mc command -- minio client. We're curious how "ls -r" is implemented
under mc. Does it need to get a full listing and then do some path
parsing to produce nice output? If so, it may be playing a role in the
delay as well.

Eric

On 9/26/18 5:27 PM, Graham Allan wrote:
> I have one user bucket, where inexplicably (to me), the bucket takes an
> eternity to list, though only on the top level. There are two
> subfolders, each of which lists individually at a completely normal
> speed...
> 
> eg (using minio client):
> 
>> [~] % time ./mc ls fried/friedlab/
>> [2018-09-26 16:15:48 CDT]     0B impute/
>> [2018-09-26 16:15:48 CDT]     0B wgs/
>>
>> real    1m59.390s
>>
>> [~] % time ./mc ls -r fried/friedlab/
>> ...
>> real 3m18.013s
>>
>> [~] % time ./mc ls -r fried/friedlab/impute
>> ...
>> real 0m13.512s
>>
>> [~] % time ./mc ls -r fried/friedlab/wgs
>> ...
>> real 0m6.437s
> 
> The bucket has about 55k objects total, with 32 index shards on a
> replicated ssd pool. It shouldn't be taking this long but I can't
> imagine what could be causing this. I haven't found any others behaving
> this way. I'd think it has to be some problem with the bucket index, but
> what...?
> 
> I did naively try some "radosgw-admin bucket check [--fix]" commands
> with no change.
> 
> Graham



More information about the ceph-users mailing list