[ceph-users] ceph zstd not for bluestor due to performance reasons

Stefan Priebe - Profihost AG s.priebe at profihost.ag
Sat Nov 4 12:41:14 PDT 2017


Hi Sage,

Am 26.10.2017 um 13:58 schrieb Sage Weil:
> On Thu, 26 Oct 2017, Stefan Priebe - Profihost AG wrote:
>> Hi Sage,
>>
>> Am 25.10.2017 um 21:54 schrieb Sage Weil:
>>> On Wed, 25 Oct 2017, Stefan Priebe - Profihost AG wrote:
>>>> Hello,
>>>>
>>>> in the lumious release notes is stated that zstd is not supported by
>>>> bluestor due to performance reason. I'm wondering why btrfs instead
>>>> states that zstd is as fast as lz4 but compresses as good as zlib.
>>>>
>>>> Why is zlib than supported by bluestor? And why does btrfs / facebook
>>>> behave different?
>>>>
>>>> "BlueStore supports inline compression using zlib, snappy, or LZ4. (Ceph
>>>> also supports zstd for RGW compression but zstd is not recommended for
>>>> BlueStore for performance reasons.)"
>>>
>>> zstd will work but in our testing the performance wasn't great for 
>>> bluestore in particular.  The problem was that for each compression run 
>>> there is a relatively high start-up cost initializing the zstd 
>>> context/state (IIRC a memset of a huge memory buffer) that dominated the 
>>> execution time... primarily because bluestore is generally compressing 
>>> pretty small chunks of data at a time, not big buffers or streams.
>>>
>>> Take a look at unittest_compression timings on compressing 16KB buffers 
>>> (smaller than bluestore needs usually, but illustrated of the problem):
>>>
>>> [ RUN      ] Compressor/CompressorTest.compress_16384/0
>>> [plugin zlib (zlib/isal)]
>>> [       OK ] Compressor/CompressorTest.compress_16384/0 (294 ms)
>>> [ RUN      ] Compressor/CompressorTest.compress_16384/1
>>> [plugin zlib (zlib/noisal)]
>>> [       OK ] Compressor/CompressorTest.compress_16384/1 (1755 ms)
>>> [ RUN      ] Compressor/CompressorTest.compress_16384/2
>>> [plugin snappy (snappy)]
>>> [       OK ] Compressor/CompressorTest.compress_16384/2 (169 ms)
>>> [ RUN      ] Compressor/CompressorTest.compress_16384/3
>>> [plugin zstd (zstd)]
>>> [       OK ] Compressor/CompressorTest.compress_16384/3 (4528 ms)
>>>
>>> It's an order of magnitude slower than zlib or snappy, which probably 
>>> isn't acceptable--even if it is a bit smaller.

i've fixed the zstd compression plugin to use reset stream instead of
initializing new objects.

What's needed to run only / just the unittest_compression test?

Stefan


More information about the ceph-users mailing list