[ceph-users] ceph-users Digest, Vol 70, Issue 23

Lazuardi Nasution mrxlazuardin at gmail.com
Sat Nov 24 16:43:30 PST 2018

Hi Robin,

Do you mean that Cumulus quagga fork is FRRouting (https://frrouting.org/)?
As long as I know Cumulus using it now. What dummy interfaces do you mean?
Why did you use it instead of loopback address? Anyway, how can you isolate
between some kind of traffic to be not routable? On L2 implementation, I
separate two Ceph traffic (public and cluster) with other traffic by using

Best regards,

Date: Fri, 23 Nov 2018 00:29:17 +0000
> From: "Robin H. Johnson" <robbat2 at gentoo.org>
> To: Ceph Users <ceph-users at lists.ceph.com>
> Subject: Re: [ceph-users] Full L3 Ceph
> Message-ID: <robbat2-20181123T001719-741076795Z at orbis-terrarum.net>
> Content-Type: text/plain; charset="us-ascii"
> On Fri, Nov 23, 2018 at 04:03:25AM +0700, Lazuardi Nasution wrote:
> > I'm looking example Ceph configuration and topology on full layer 3
> > networking deployment. Maybe all daemons can use loopback alias address
> in
> > this case. But how to set cluster network and public network
> configuration,
> > using supernet? I think using loopback alias address can prevent the
> > daemons down due to physical interfaces disconnection and can load
> balance
> > traffic between physical interfaces without interfaces bonding, but with
> > ECMP.
> I can say I've done something similar**, but I don't have access to that
> environment or most*** of the configuration anymore.
> One of the parts I do recall, was explicitly setting cluster_network
> and public_network to empty strings, AND using public_addr+cluster_addr
> instead, with routable addressing on dummy interfaces (NOT loopback).
> **:For values of similar:
> - 99.9% IPv6 environment
> - BGP everywhere
> - The only IPv4 was on the outside of HAProxy for legacy IPv4 clients.
> - Quanta switchgear running Cumulus Linux, 10Gbit ports
> - Hosts running Cumulus quagga fork (REQUIRED)
> - Host to 2xToR using IPv6 link-local addressing only
>   https://blog.ipspace.net/2015/02/bgp-configuration-made-simple-with.html
> - Reliable ~19Gbit aggregate (2x10GBit)
> - watch out for NIC overheating: no warning, just thermal throttle down
>   to ~2.5Gbit/port.
> ***:Some parts of the configuration ARE public:
> https://github.com/dreamhost/ceph-chef/tree/dokken
> --
> Robin Hugh Johnson
> Gentoo Linux: Dev, Infra Lead, Foundation Treasurer
> E-Mail   : robbat2 at gentoo.org
> GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
> GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ceph.com/pipermail/ceph-users-ceph.com/attachments/20181125/76034cc0/attachment.html>

More information about the ceph-users mailing list