[ceph-users] ceph zstd not for bluestor due to performance reasons

Sage Weil sage at newdream.net
Sat Nov 4 13:10:54 PDT 2017


On Sat, 4 Nov 2017, Stefan Priebe - Profihost AG wrote:
> Hi Sage,
> 
> Am 26.10.2017 um 13:58 schrieb Sage Weil:
> > On Thu, 26 Oct 2017, Stefan Priebe - Profihost AG wrote:
> >> Hi Sage,
> >>
> >> Am 25.10.2017 um 21:54 schrieb Sage Weil:
> >>> On Wed, 25 Oct 2017, Stefan Priebe - Profihost AG wrote:
> >>>> Hello,
> >>>>
> >>>> in the lumious release notes is stated that zstd is not supported by
> >>>> bluestor due to performance reason. I'm wondering why btrfs instead
> >>>> states that zstd is as fast as lz4 but compresses as good as zlib.
> >>>>
> >>>> Why is zlib than supported by bluestor? And why does btrfs / facebook
> >>>> behave different?
> >>>>
> >>>> "BlueStore supports inline compression using zlib, snappy, or LZ4. (Ceph
> >>>> also supports zstd for RGW compression but zstd is not recommended for
> >>>> BlueStore for performance reasons.)"
> >>>
> >>> zstd will work but in our testing the performance wasn't great for 
> >>> bluestore in particular.  The problem was that for each compression run 
> >>> there is a relatively high start-up cost initializing the zstd 
> >>> context/state (IIRC a memset of a huge memory buffer) that dominated the 
> >>> execution time... primarily because bluestore is generally compressing 
> >>> pretty small chunks of data at a time, not big buffers or streams.
> >>>
> >>> Take a look at unittest_compression timings on compressing 16KB buffers 
> >>> (smaller than bluestore needs usually, but illustrated of the problem):
> >>>
> >>> [ RUN      ] Compressor/CompressorTest.compress_16384/0
> >>> [plugin zlib (zlib/isal)]
> >>> [       OK ] Compressor/CompressorTest.compress_16384/0 (294 ms)
> >>> [ RUN      ] Compressor/CompressorTest.compress_16384/1
> >>> [plugin zlib (zlib/noisal)]
> >>> [       OK ] Compressor/CompressorTest.compress_16384/1 (1755 ms)
> >>> [ RUN      ] Compressor/CompressorTest.compress_16384/2
> >>> [plugin snappy (snappy)]
> >>> [       OK ] Compressor/CompressorTest.compress_16384/2 (169 ms)
> >>> [ RUN      ] Compressor/CompressorTest.compress_16384/3
> >>> [plugin zstd (zstd)]
> >>> [       OK ] Compressor/CompressorTest.compress_16384/3 (4528 ms)
> >>>
> >>> It's an order of magnitude slower than zlib or snappy, which probably 
> >>> isn't acceptable--even if it is a bit smaller.
> 
> i've fixed the zstd compression plugin to use reset stream instead of
> initializing new objects.
> 
> What's needed to run only / just the unittest_compression test?

make unittest_compression && bin/unittest_compression

should do it!

sage


More information about the ceph-users mailing list