[ceph-users] injecting args output misleading

Gregory Farnum gfarnum at redhat.com
Mon Dec 4 13:38:58 PST 2017


On Mon, Dec 4, 2017 at 12:12 PM Brady Deetz <bdeetz at gmail.com> wrote:

> I'm not sure if this is a bug where ceph incorrectly reports to the user
> or if this is just a matter of misleading language. Thought I might bring
> it up in any case.
>
> I under stand that "may require restart" is fairly direct in its
> ambiguity, but this probably shouldn't be ambiguous without a good
> technical reason. But, I find "not observed" to be quite misleading. These
> arg injections are very clearly being observed. Maybe the output should be
> "not observed by 'component x', change may require restart." But, I'd still
> like a definitive yes or no for service restarts required by arg injects.
>

You're right, but unfortunately there is a good technical reason. Or at
least one of effort versus payoff.

We are slowly (or, lately, somewhat rapidly) changing things in
configuration management within Ceph, but the historical pattern is that we
have a giant struct of variables, and when we want a new config option we
add a new variable. These are initialized on startup.

Initially, whenever we wanted to access the variables, we just read them
directly out of the config struct. Unfortunately, this could be racy in the
case where we changed configs while running, and some configs can't be
changed live. So we added an "observer" framework, which lets us call
specific functions whenever specified config options get changed, and a set
of thread-safe accessor functions.

Then we saw people injecting options we know don't take live effect, so we
wanted to add a warning. (For instance, you can't change the number of
worker shards in the OSD once it's running.) Obviously, we can assume
programmatically that any configuration option with an observer will take
live effect. But what about places where we still read the value directly
out of the config struct while running? Those *do* take effect immediately,
but there's no programmatic way to identify them.

We've talked in the past about adding an extra flag which indicates if the
config is a live-changed one or not, but it's run up against issues of
difficulty and the effort required to track down every option. (Adding yet
another "this might not be registered" statement won't help us much, so it
basically needs to be done monolithically.) John recently reworked the
config options framework and it may be easier to make these changes now,
but the effort of auditing every single option remains. :/

Although it could be that even if we don't know for sure, being able to say
with certainty that the ten most-common unobserved options *will* take live
effect is useful enough on its own. Patches welcome! ;)
-Greg


>
> I've run into this on osd args as well.
>
> Ceph Luminous 12.2.1 (CentOS 7.4.1708)
>
> [root at mon0 ceph-admin]# ceph --admin-daemon
> /var/run/ceph/ceph-mon.mon0.asok config show | grep "mon_allow_pool_delete"
>     "mon_allow_pool_delete": "false",
>
> [root at mon0 ceph-admin]# ceph tell mon.0 injectargs
> '--mon_allow_pool_delete=true'
> injectargs:mon_allow_pool_delete = 'true' (not observed, change may
> require restart)
> [root at mon0 ceph-admin]# ceph tell mon.1 injectargs
> '--mon_allow_pool_delete=true'
> injectargs:mon_allow_pool_delete = 'true' (not observed, change may
> require restart)
> [root at mon0 ceph-admin]# ceph tell mon.2 injectargs
> '--mon_allow_pool_delete=true'
> injectargs:mon_allow_pool_delete = 'true' (not observed, change may
> require restart)
>
> [root at mon0 ceph-admin]# ceph --admin-daemon
> /var/run/ceph/ceph-mon.mon0.asok config show | grep "mon_allow_pool_delete"
>     "mon_allow_pool_delete": "true",
>
> [root at mon0 ceph-admin]# ceph tell mon.0 injectargs
> '--mon_allow_pool_delete=false'
> injectargs:mon_allow_pool_delete = 'false' (not observed, change may
> require restart)
> [root at mon0 ceph-admin]# ceph tell mon.1 injectargs
> '--mon_allow_pool_delete=false'
> injectargs:mon_allow_pool_delete = 'false' (not observed, change may
> require restart)
> [root at mon0 ceph-admin]# ceph tell mon.2 injectargs
> '--mon_allow_pool_delete=false'
> injectargs:mon_allow_pool_delete = 'false' (not observed, change may
> require restart)
>
> [root at mon0 ceph-admin]# ceph --admin-daemon
> /var/run/ceph/ceph-mon.mon0.asok config show | grep "mon_allow_pool_delete"
>     "mon_allow_pool_delete": "false",
>
> Thanks for the hard work, devs!
> _______________________________________________
> ceph-users mailing list
> ceph-users at lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ceph.com/pipermail/ceph-users-ceph.com/attachments/20171204/04d183be/attachment.html>


More information about the ceph-users mailing list