[ceph-users] Bluestore performance 50% of filestore

Milanov, Radoslav Nikiforov radonm at bu.edu
Tue Nov 14 11:54:29 PST 2017

We have 3 node, 27 OSDs cluster running Luminous 12.2.1
In filestore configuration there are 3 SSDs used for journals of 9 OSDs on each hosts (1 SSD has 3 journal paritions for 3 OSDs).
I've converted filestore to bluestore by wiping 1 host a time and waiting for recovery. SSDs now contain block-db - again one SSD serving 3 OSDs.

Cluster is used as storage for Openstack.
Running fio on a VM in that Openstack reveals bluestore performance almost twice slower than filestore.
fio --name fio_test_file --direct=1 --rw=randwrite --bs=4k --size=1G --numjobs=2 --time_based --runtime=180 --group_reporting
fio --name fio_test_file --direct=1 --rw=randread --bs=4k --size=1G --numjobs=2 --time_based --runtime=180 --group_reporting


  write: io=3511.9MB, bw=19978KB/s, iops=4994, runt=180001msec

  write: io=3525.6MB, bw=20057KB/s, iops=5014, runt=180001msec

  write: io=3554.1MB, bw=20222KB/s, iops=5055, runt=180016msec

  read : io=1995.7MB, bw=11353KB/s, iops=2838, runt=180001msec

  read : io=1824.5MB, bw=10379KB/s, iops=2594, runt=180001msec

  read : io=1966.5MB, bw=11187KB/s, iops=2796, runt=180001msec


  write: io=1621.2MB, bw=9222.3KB/s, iops=2305, runt=180002msec

  write: io=1576.3MB, bw=8965.6KB/s, iops=2241, runt=180029msec

  write: io=1531.9MB, bw=8714.3KB/s, iops=2178, runt=180001msec

  read : io=1279.4MB, bw=7276.5KB/s, iops=1819, runt=180006msec

  read : io=773824KB, bw=4298.9KB/s, iops=1074, runt=180010msec

  read : io=1018.5MB, bw=5793.7KB/s, iops=1448, runt=180001msec

- Rado

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ceph.com/pipermail/ceph-users-ceph.com/attachments/20171114/96c04a31/attachment.html>

More information about the ceph-users mailing list