[ceph-users] optimize bluestore for random write i/o

vitalif at yourcmc.ru vitalif at yourcmc.ru
Tue Mar 12 07:51:02 PDT 2019


> I bet you'd see better memstore results with my vector based object
> implementation instead of bufferlists.

Where can I find it?

> Nick Fisk noticed the same
> thing you did.  One interesting observation he made was that disabling
> CPU C/P states helped bluestore immensely in the iodepth=1 case.

This is exactly what I've done by `cpupower idle-set -D 0`. It basically 
increases iops 2-3 times.

> Pipelined writes were added in rocksdb 5.5.1 back in the summer of 
> 2017.  That wasn't available when bluestore was being written.

In fact ... it slightly confuses me because even now bluestore IS 
writing to rocksdb from multiple threads sometimes. It's when 
bluestore_sync_submit_transaction is on and the write doesn't require 
aio (= when it's a deferred write) and when it holds several other 
conditions. It calls db->submit_transaction from the tp_osd_tp thread 
then.


More information about the ceph-users mailing list