[ceph-users] osd max write size and large objects

Wido den Hollander wido at 42on.com
Thu Nov 16 08:05:20 PST 2017


> Op 16 november 2017 om 16:46 schreef Robert Stanford <rstanford8896 at gmail.com>:
> 
> 
>  Thanks.  Does this mean that I can send >90MB objects to my RGWs and they
> will break it up into manageable (<=90MB) chunks before storing them?  Or
> if I'm going to store objects > 90MB do I need to change this parameter?
> 

Yes, the RGW breaks it up into 4MB chunks and writes that to RADOS.

In your case, don't worry about this, upload a 100MB object to RGW and it will take care of the rest.

>  I don't know if we'll be able to use libradosstriper, but thanks for
> bringing it to my attention.  We are using the S3 interface of RGW
> exclusively (nothing custom in there).
> 

Only a custom application will use libradosstriper. So don't worry.

Wido

> 
> On Thu, Nov 16, 2017 at 9:41 AM, Wido den Hollander <wido at 42on.com> wrote:
> 
> >
> > > Op 16 november 2017 om 16:32 schreef Robert Stanford <
> > rstanford8896 at gmail.com>:
> > >
> > >
> > >  Once 'osd max write size' (90MB by default I believe) is exceeded, does
> > > Ceph reject the object (which is coming in through RGW), or does it break
> > > it up into smaller objects (of max 'osd  max write size' size)?  If it
> > > breaks them up, does it read the fragments in parallel when they're
> > > requested by RGW?
> > >
> >
> > The OSD rejects the write. However, RGW already does striping over RADOS
> > objects.
> >
> > Usually you don't need to worry about this setting unless you are talking
> > to RADOS directly.
> >
> > However, 90MB objects are big and you should look into libradosstriper.
> >
> > Wido
> >
> > >  Thanks
> > > _______________________________________________
> > > ceph-users mailing list
> > > ceph-users at lists.ceph.com
> > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> >


More information about the ceph-users mailing list