[ceph-users] ceph df space usage confusion - balancing needed?

Oliver Freyermuth freyermuth at physik.uni-bonn.de
Sat Oct 20 11:28:32 PDT 2018


Dear Janne,

yes, of course. But since we only have two pools here, this can not explain the difference. 
The metadata is replicated (3 copies) across ssd drives, and we have < 3 TB of total raw storage for that. 
So looking at the raw space usage, we can ignore that. 

All the rest is used for the ceph_data pool. So the ceph_data pool, in terms of raw storage, is about 50 % used. 

But in terms of storage shown for that pool, it's almost 63 % %USED. 
So I guess this can purely be from bad balancing, correct? 

Cheers,
	Oliver

Am 20.10.18 um 19:49 schrieb Janne Johansson:
> Do mind that drives may have more than one pool on them, so RAW space
> is what it says, how much free space there is. Then the avail and
> %USED on per-pool stats will take replication into account, it can
> tell how much data you may write into that particular pool, given that
> pools replication or EC settings.
> 
> Den lör 20 okt. 2018 kl 19:09 skrev Oliver Freyermuth
> <freyermuth at physik.uni-bonn.de>:
>>
>> Dear Cephalopodians,
>>
>> as many others, I'm also a bit confused by "ceph df" output
>> in a pretty straightforward configuration.
>>
>> We have a CephFS (12.2.7) running, with 4+2 EC profile.
>>
>> I get:
>> ----------------------------------------------------------------------------
>> # ceph df
>> GLOBAL:
>>     SIZE     AVAIL     RAW USED     %RAW USED
>>     824T      410T         414T         50.26
>> POOLS:
>>     NAME                ID     USED     %USED     MAX AVAIL     OBJECTS
>>     cephfs_metadata     1      452M      0.05          860G       365774
>>     cephfs_data         2      275T     62.68          164T     75056403
>> ----------------------------------------------------------------------------
>>
>> So about 50 % of raw space are used, but already ~63 % of filesystem space are used.
>> Is this purely from imperfect balancing?
>> In "ceph osd df", I do indeed see OSD usages spreading from 65.02 % usage down to 37.12 %.
>>
>> We did not yet use the balancer plugin.
>> We don't have any pre-luminous clients.
>> In that setup, I take it that "upmap" mode would be recommended - correct?
>> Any "gotchas" using that on luminous?
>>
>> Cheers,
>>         Oliver
>>
>> _______________________________________________
>> ceph-users mailing list
>> ceph-users at lists.ceph.com
>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> 
> 
> 


-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5455 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.ceph.com/pipermail/ceph-users-ceph.com/attachments/20181020/f8751935/attachment.bin>


More information about the ceph-users mailing list