[ceph-users] RBD Mirror Question

Vikas Rana vikasrana3 at gmail.com
Thu Oct 4 07:27:15 PDT 2018


on Primary site, we have OSD's running on 192.168.4.x address.

Similarly on Secondary site, we have OSD's running on 192.168.4.x address.
192.168.3.x is the old MON network.on both site which was non route-able.
So we renamed mon on primary site to 165.x.x and mon on secondary site to
165.x.y. now primary and secondary can see each other.


Do the OSD daemon from primary and secondary have to talk to each other? we
have same non routed networks for OSD.

Thanks,
-Vikas

On Thu, Oct 4, 2018 at 10:13 AM Jason Dillaman <jdillama at redhat.com> wrote:

> On Thu, Oct 4, 2018 at 10:10 AM Vikas Rana <vikasrana3 at gmail.com> wrote:
> >
> > Thanks Jason for great suggestions.
> >
> > but somehow rbd mirror status not working from secondary to primary.
> Here;s the status from both sides. cluster name is ceph on primary side and
> cephdr on remote site. mirrordr is the user on DR side and mirrorprod is on
> primary prod side.
> >
> > # rbd mirror pool info nfs
> > Mode: image
> > Peers:
> >   UUID                                 NAME   CLIENT
> >   3ccd7a67-2343-44bf-960b-02d9b1258371 cephdr client.mirrordr.
> >
> > rbd --cluster cephdr mirror pool info nfs
> > Mode: image
> > Peers:
> >   UUID                                 NAME CLIENT
> >   e6b9ba05-48de-462c-ad5f-0b51d0ee733f ceph client.mirrorprod
> >
> >
> > From primary site, when i query the remote site, its looks good.
> > # rbd --cluster cephdr --id mirrordr mirror pool status nfs
> > health: OK
> > images: 0 total
> >
> > but when i query from secondary site to primary side, I'm getting this
> error
> > # rbd  --cluster ceph --id mirrorprod mirror pool status nfs
> > 2018-10-03 10:21:06.645903 7f27a44ed700  0 -- 165.x.x.202:0/1310074448
> >> 192.168.3.21:6804/3835 pipe(0x55ed47daf480 sd=4 :0 s=1 pgs=0 cs=0 l=1
> c=0x55ed47db0740).fault
> >
> >
> > We were using 192.168.3.x for MON network before we renamed it to use
> 165 address since its routeable. why its trying to connect to 192.x address
> instead of 165.x.y address?
>
> Are your OSDs on that 192.168.3.x subnet? What daemons are running on
> 192.168.3.21?
>
> > I could do ceph -s from both side and they can see each other. Only rbd
> command is having issue.
> >
> > Thanks,
> > -Vikas
> >
> >
> >
> >
> > On Tue, Oct 2, 2018 at 5:14 PM Jason Dillaman <jdillama at redhat.com>
> wrote:
> >>
> >> On Tue, Oct 2, 2018 at 4:47 PM Vikas Rana <vikasrana3 at gmail.com> wrote:
> >> >
> >> > Hi,
> >> >
> >> > We have a CEPH 3 node cluster at primary site. We created a RBD image
> and the image has about 100TB of data.
> >> >
> >> > Now we installed another 3 node cluster on secondary site. We want to
> replicate the image at primary site to this new cluster on secondary site.
> >> >
> >> > As per documentation, we enabled journaling on primary site. We
> followed all the procedure and peering looks good but the image is not
> copying.
> >> > The status is always showing down.
> >>
> >> Do you have an "rbd-mirror" daemon running on the secondary site? Are
> >> you running "rbd mirror pool status" against the primary site or the
> >> secondary site? The mirroring status is only available on the sites
> >> running "rbd-mirror" daemon (the "down" means that the cluster you are
> >> connected to doesn't have the daemon running).
> >>
> >> > So my question is, is it possible to replicate a image which already
> have some data before enabling journalling?
> >>
> >> Indeed -- it will perform a full image sync to the secondary site.
> >>
> >> > We are using the image mirroring instead of pool mirroring. Do we
> need to create the RBD image on secondary site? As per documentation, its
> not required.
> >>
> >> The only difference between the two modes is whether or not you need
> >> to run "rbd mirror image enable" or not.
> >>
> >> > Is there any other option to copy the image to the remote site?
> >>
> >> No other procedure should be required.
> >>
> >> > Thanks,
> >> > -Vikas
> >> > _______________________________________________
> >> > ceph-users mailing list
> >> > ceph-users at lists.ceph.com
> >> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> >>
> >>
> >>
> >> --
> >> Jason
>
>
>
> --
> Jason
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ceph.com/pipermail/ceph-users-ceph.com/attachments/20181004/057c1af9/attachment.html>


More information about the ceph-users mailing list