[ceph-users] compacting omap doubles its size

Tomasz Płaza tomasz.plaza at grupawp.pl
Wed Nov 28 22:49:41 PST 2018


I have a ceph 12.2.8 cluster on filestore with rather large omap dirs 
(avg size is about 150G). Recently slow requests became a problem, so 
after some digging I decided to convert omap from leveldb to rocksdb. 
Conversion went fine and slow requests rate went down to acceptable 
level. Unfortunately  conversion did not shrink most of omap dirs, so I 
tried online compaction:

Before compaction: 50G    /var/lib/ceph/osd/ceph-0/current/omap/

After compaction: 100G    /var/lib/ceph/osd/ceph-0/current/omap/

Purge and recreate: 1.5G /var/lib/ceph/osd/ceph-0/current/omap/

Before compaction: 135G    /var/lib/ceph/osd/ceph-5/current/omap/

After compaction: 260G    /var/lib/ceph/osd/ceph-5/current/omap/

Purge and recreate: 2.5G /var/lib/ceph/osd/ceph-5/current/omap/

For me compaction which makes omap bigger is quite weird and 
frustrating. Please help.

P.S. My cluster suffered from ongoing index reshards (it is disabled 
now) and on many buckets with 4m+ objects I have a lot of old indexes:

634   bucket1
651   bucket2

1231 bucket17
1363 bucket18

More information about the ceph-users mailing list