[ceph-users] optimize bluestore for random write i/o

Mark Nelson mnelson at redhat.com
Tue Mar 12 06:50:02 PDT 2019


On 3/12/19 8:40 AM, vitalif at yourcmc.ru wrote:
>> One way or another we can only have a single thread sending writes to
>> rocksdb.  A lot of the prior optimization work on the write side was
>> to get as much processing out of the kv_sync_thread as possible.
>> That's still a worthwhile goal as it's typically what bottlenecks with
>> high amounts of concurrency.  What I think would be very interesting
>> though is if we moved more toward a model where we had lots shards
>> (OSDs or shards of an OSD) with independent rocksdb instances and less
>> threading overhead per shard.  That's the way the seastar work is
>> going, and also sort of the model I've been thinking about for a very
>> simple single-threaded OSD.
>
> Doesn't rocksdb have pipelined writes? Isn't it better to just use 
> that builtin concurrency instead of factoring in your own?


Pipelined writes were added in rocksdb 5.5.1 back in the summer of 
2017.  That wasn't available when bluestore was being written. We may be 
able to make use of it now but I don't think anyone has taken the time 
to figure out how much work it would take or what kind of benefit we 
would get.


Mark



More information about the ceph-users mailing list