[ceph-users] ceph zstd not for bluestor due to performance reasons
Stefan Priebe - Profihost AG
s.priebe at profihost.ag
Sat Nov 4 12:41:14 PDT 2017
Am 26.10.2017 um 13:58 schrieb Sage Weil:
> On Thu, 26 Oct 2017, Stefan Priebe - Profihost AG wrote:
>> Hi Sage,
>> Am 25.10.2017 um 21:54 schrieb Sage Weil:
>>> On Wed, 25 Oct 2017, Stefan Priebe - Profihost AG wrote:
>>>> in the lumious release notes is stated that zstd is not supported by
>>>> bluestor due to performance reason. I'm wondering why btrfs instead
>>>> states that zstd is as fast as lz4 but compresses as good as zlib.
>>>> Why is zlib than supported by bluestor? And why does btrfs / facebook
>>>> behave different?
>>>> "BlueStore supports inline compression using zlib, snappy, or LZ4. (Ceph
>>>> also supports zstd for RGW compression but zstd is not recommended for
>>>> BlueStore for performance reasons.)"
>>> zstd will work but in our testing the performance wasn't great for
>>> bluestore in particular. The problem was that for each compression run
>>> there is a relatively high start-up cost initializing the zstd
>>> context/state (IIRC a memset of a huge memory buffer) that dominated the
>>> execution time... primarily because bluestore is generally compressing
>>> pretty small chunks of data at a time, not big buffers or streams.
>>> Take a look at unittest_compression timings on compressing 16KB buffers
>>> (smaller than bluestore needs usually, but illustrated of the problem):
>>> [ RUN ] Compressor/CompressorTest.compress_16384/0
>>> [plugin zlib (zlib/isal)]
>>> [ OK ] Compressor/CompressorTest.compress_16384/0 (294 ms)
>>> [ RUN ] Compressor/CompressorTest.compress_16384/1
>>> [plugin zlib (zlib/noisal)]
>>> [ OK ] Compressor/CompressorTest.compress_16384/1 (1755 ms)
>>> [ RUN ] Compressor/CompressorTest.compress_16384/2
>>> [plugin snappy (snappy)]
>>> [ OK ] Compressor/CompressorTest.compress_16384/2 (169 ms)
>>> [ RUN ] Compressor/CompressorTest.compress_16384/3
>>> [plugin zstd (zstd)]
>>> [ OK ] Compressor/CompressorTest.compress_16384/3 (4528 ms)
>>> It's an order of magnitude slower than zlib or snappy, which probably
>>> isn't acceptable--even if it is a bit smaller.
i've fixed the zstd compression plugin to use reset stream instead of
initializing new objects.
What's needed to run only / just the unittest_compression test?
More information about the ceph-users