[ceph-users] Intel Xeon Scalable and CPU frequency scaling on NVMe/SSD Ceph OSDs

Wido den Hollander wido at 42on.com
Tue May 1 08:07:45 PDT 2018


Hi,

I've been trying to get the lowest latency possible out of the new Xeon
Scalable CPUs and so far I got down to 1.3ms with the help of Nick.

However, I can't seem to pin the CPUs to always run at their maximum
frequency.

If I disable power saving in the BIOS they stay at 2.1Ghz (Silver 4110),
but that disables the boost.

With the Power Saving enabled in the BIOS and when giving the OS all
control for some reason the CPUs keep scaling down.

$ echo 100 > /sys/devices/system/cpu/intel_pstate/min_perf_pct

cpufrequtils 008: cpufreq-info (C) Dominik Brodowski 2004-2009
Report errors and bugs to cpufreq at vger.kernel.org, please.
analyzing CPU 0:
  driver: intel_pstate
  CPUs which run at the same hardware frequency: 0
  CPUs which need to have their frequency coordinated by software: 0
  maximum transition latency: 0.97 ms.
  hardware limits: 800 MHz - 3.00 GHz
  available cpufreq governors: performance, powersave
  current policy: frequency should be within 800 MHz and 3.00 GHz.
                  The governor "performance" may decide which speed to use
                  within this range.
  current CPU frequency is 800 MHz.

I do see the CPUs scale up to 2.1Ghz, but they quickly scale down again
to 800Mhz and that hurts latency. (50% difference!)

With the CPUs scaling down to 800Mhz my latency jumps from 1.3ms to
2.4ms on avg. With turbo enabled I hope to get down to 1.1~1.2ms on avg.

$ cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
performance

Everything seems to be OK and I would expect the CPUs to stay at
2.10Ghz, but they aren't.

C-States are also pinned to 0 as a boot parameter for the kernel:

processor.max_cstate=1 intel_idle.max_cstate=0

Running Ubuntu 16.04.4 with the 4.13 kernel from the HWE from Ubuntu.

Has anybody tried this yet with the recent Intel Xeon Scalable CPUs?

Thanks,

Wido


More information about the ceph-users mailing list