[ceph-users] Full L3 Ceph

Robin H. Johnson robbat2 at gentoo.org
Thu Nov 22 16:29:17 PST 2018


On Fri, Nov 23, 2018 at 04:03:25AM +0700, Lazuardi Nasution wrote:
> I'm looking example Ceph configuration and topology on full layer 3
> networking deployment. Maybe all daemons can use loopback alias address in
> this case. But how to set cluster network and public network configuration,
> using supernet? I think using loopback alias address can prevent the
> daemons down due to physical interfaces disconnection and can load balance
> traffic between physical interfaces without interfaces bonding, but with
> ECMP.
I can say I've done something similar**, but I don't have access to that
environment or most*** of the configuration anymore.

One of the parts I do recall, was explicitly setting cluster_network
and public_network to empty strings, AND using public_addr+cluster_addr
instead, with routable addressing on dummy interfaces (NOT loopback).

**:For values of similar:
- 99.9% IPv6 environment
- BGP everywhere
- The only IPv4 was on the outside of HAProxy for legacy IPv4 clients.
- Quanta switchgear running Cumulus Linux, 10Gbit ports
- Hosts running Cumulus quagga fork (REQUIRED)
- Host to 2xToR using IPv6 link-local addressing only
  https://blog.ipspace.net/2015/02/bgp-configuration-made-simple-with.html
- Reliable ~19Gbit aggregate (2x10GBit)
- watch out for NIC overheating: no warning, just thermal throttle down
  to ~2.5Gbit/port.

***:Some parts of the configuration ARE public:
https://github.com/dreamhost/ceph-chef/tree/dokken

-- 
Robin Hugh Johnson
Gentoo Linux: Dev, Infra Lead, Foundation Treasurer
E-Mail   : robbat2 at gentoo.org
GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 1113 bytes
Desc: Digital signature
URL: <http://lists.ceph.com/pipermail/ceph-users-ceph.com/attachments/20181123/d6530ddf/attachment.sig>


More information about the ceph-users mailing list