[ceph-users] CephFS log jam prevention

Yan, Zheng ukernel at gmail.com
Thu Dec 7 17:24:44 PST 2017


On Thu, Dec 7, 2017 at 11:59 PM, Reed Dier <reed.dier at focusvq.com> wrote:
>> You can try doubling (several times if necessary) the MDS configs
>> `mds_log_max_segments` and `mds_log_max_expiring` to make it more
>> aggressively trim its journal. (That may not help since your OSD
>> requests are slow.)
>
>
> This may be obvious, but where is this mds_log located, and what are the bottlenecks for it to get behind?
>
> Is this something that is located on an OSD? Is this located in the metadata pool (which I had previously moved to live on SSDs rather than colocated on the HDDs that the filesystem pool lives on)?

logs are stored in cephfs metadata pool

> Just curious what would be the bottleneck in the MDS trying to trim this log.
>

see https://github.com/ceph/ceph/pull/18783. there was bug in log
trimming log. By default, it trim 20 segments at most in each tick (5
seconds by default)

> I see the MDS process on the active MDS with decent CPU util, but not any real disk traffic from the MDS side.
>
> So just looking to see what is bounding me with the trimming to keep it so far behind (I increased both max segments and max expiring by 4x each).
>
> Thanks,
>
> Reed
>


More information about the ceph-users mailing list