[ceph-users] Compression never better than 50%

Ragan, Tj (Dr.) tj.ragan at leicester.ac.uk
Wed Mar 13 04:55:15 PDT 2019

Hi All,

I’ve been investigating compression and (long-story-short,) found that I can never get better than 50% compression ratio.

My setup:
Mimic 13.2.2
OSDs:  Bluestore, Sparse files looped to /dev/loop0, lvm to create logical volumes.  bluestore_compression_mode: passive
Pool: 3-replica, compression_mode: force, compression_algorithm: zstd
Data: dd if=/dev/zero of=dd_test bs=2M count=1

Before addition to rados there are zero objects in the pool and, perf dump of one of the osds contains:
        "bluestore_allocated": 165281792,
        "bluestore_stored": 54092500,
        "bluestore_compressed": 0,
        "bluestore_compressed_allocated": 0,
        "bluestore_compressed_original": 0,
And after `sudo rados -p rep3_testing put dd_test dd_test`, `rados ls` shows only the one object, and the same osd as above contains:
        "bluestore_allocated": 166330368,
        "bluestore_stored": 56189652,
        "bluestore_compressed": 592,
        "bluestore_compressed_allocated": 1048576,
        "bluestore_compressed_original": 2097152,

… which means that the bluestore_stored went up by 2097152 bytes (exactly the file size of dd_test) and the bluestore_stored went up by 1048576 - which is exactly half as much.  (Note that these numbers are the same as the bluestore_compressed_* values.)  When I repeated this test using zlib or snappy, I got exactly 50% compression with both as well.   If instead I use the zstd command line program, this file ends up being just 207 bytes - which is what you’d expect for a file of just zeros.

Please can explain why I'm not getting better compression, or ideally tell me I’ve forgotten a setting and which setting I need to change?

Thank you,

-TJ Ragan

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ceph.com/pipermail/ceph-users-ceph.com/attachments/20190313/ec059d8b/attachment.html>

More information about the ceph-users mailing list