[ceph-users] ceph zstd not for bluestor due to performance reasons

Stefan Priebe - Profihost AG s.priebe at profihost.ag
Sat Nov 4 14:21:49 PDT 2017


OK zstd.h needs:
#define ZSTD_STATIC_LINKING_ONLY

to export that one.

Stefan

Am 04.11.2017 um 21:23 schrieb Stefan Priebe - Profihost AG:
> Thanks - not a C++ guy what's wrong with it?
> 
> /build/ceph/src/compressor/zstd/ZstdCompressor.h: In member function
> 'virtual int ZstdCompressor::compress(const bufferlist&,
> ceph::bufferlist&)':
> /build/ceph/src/compressor/zstd/ZstdCompressor.h:48:28: error:
> 'ZSTD_resetCStream' was not declared in this scope
>      ZSTD_resetCStream(cs, 0);
>                             ^
> /build/ceph/src/compressor/zstd/ZstdCompressor.h: In member function
> 'virtual int ZstdCompressor::decompress(ceph::buffer::list::iterator&,
> size_t, ceph::bufferlist&)':
> /build/ceph/src/compressor/zstd/ZstdCompressor.h:89:28: error:
> 'ZSTD_resetDStream' was not declared in this scope
>      ZSTD_resetDStream(ds, 0);
> 
> 
> zstd.h which is included has the function:
> 
> src/zstd/lib/zstd.h:ZSTDLIB_API size_t ZSTD_resetCStream(ZSTD_CStream*
> zcs, unsigned long long pledgedSrcSize);  /**< re-use compression
> parameters from previous init; skip dictionary loading stage; zcs must
> be init at least once before */
> 
> Stefan
> 
> Am 04.11.2017 um 21:10 schrieb Sage Weil:
>> On Sat, 4 Nov 2017, Stefan Priebe - Profihost AG wrote:
>>> Hi Sage,
>>>
>>> Am 26.10.2017 um 13:58 schrieb Sage Weil:
>>>> On Thu, 26 Oct 2017, Stefan Priebe - Profihost AG wrote:
>>>>> Hi Sage,
>>>>>
>>>>> Am 25.10.2017 um 21:54 schrieb Sage Weil:
>>>>>> On Wed, 25 Oct 2017, Stefan Priebe - Profihost AG wrote:
>>>>>>> Hello,
>>>>>>>
>>>>>>> in the lumious release notes is stated that zstd is not supported by
>>>>>>> bluestor due to performance reason. I'm wondering why btrfs instead
>>>>>>> states that zstd is as fast as lz4 but compresses as good as zlib.
>>>>>>>
>>>>>>> Why is zlib than supported by bluestor? And why does btrfs / facebook
>>>>>>> behave different?
>>>>>>>
>>>>>>> "BlueStore supports inline compression using zlib, snappy, or LZ4. (Ceph
>>>>>>> also supports zstd for RGW compression but zstd is not recommended for
>>>>>>> BlueStore for performance reasons.)"
>>>>>>
>>>>>> zstd will work but in our testing the performance wasn't great for 
>>>>>> bluestore in particular.  The problem was that for each compression run 
>>>>>> there is a relatively high start-up cost initializing the zstd 
>>>>>> context/state (IIRC a memset of a huge memory buffer) that dominated the 
>>>>>> execution time... primarily because bluestore is generally compressing 
>>>>>> pretty small chunks of data at a time, not big buffers or streams.
>>>>>>
>>>>>> Take a look at unittest_compression timings on compressing 16KB buffers 
>>>>>> (smaller than bluestore needs usually, but illustrated of the problem):
>>>>>>
>>>>>> [ RUN      ] Compressor/CompressorTest.compress_16384/0
>>>>>> [plugin zlib (zlib/isal)]
>>>>>> [       OK ] Compressor/CompressorTest.compress_16384/0 (294 ms)
>>>>>> [ RUN      ] Compressor/CompressorTest.compress_16384/1
>>>>>> [plugin zlib (zlib/noisal)]
>>>>>> [       OK ] Compressor/CompressorTest.compress_16384/1 (1755 ms)
>>>>>> [ RUN      ] Compressor/CompressorTest.compress_16384/2
>>>>>> [plugin snappy (snappy)]
>>>>>> [       OK ] Compressor/CompressorTest.compress_16384/2 (169 ms)
>>>>>> [ RUN      ] Compressor/CompressorTest.compress_16384/3
>>>>>> [plugin zstd (zstd)]
>>>>>> [       OK ] Compressor/CompressorTest.compress_16384/3 (4528 ms)
>>>>>>
>>>>>> It's an order of magnitude slower than zlib or snappy, which probably 
>>>>>> isn't acceptable--even if it is a bit smaller.
>>>
>>> i've fixed the zstd compression plugin to use reset stream instead of
>>> initializing new objects.
>>>
>>> What's needed to run only / just the unittest_compression test?
>>
>> make unittest_compression && bin/unittest_compression
>>
>> should do it!
>>
>> sage
>>


More information about the ceph-users mailing list