[ceph-users] Bareos and libradosstriper works only for 4M sripe_unit size

Maged Mokhtar mmokhtar at petasan.org
Tue Oct 17 00:00:00 PDT 2017


>> Would it be 4 objects of 24M and 4 objects of 250KB? Or will the last
4 objects be artificially padded (with 0's) to meet the stripe_unit? 

It will be 4 object of 24M + 1M stored on the 5th object 

If you write 104M :  4 object of 24M + 8M stored on the 5th object 

If you write 105M :  4 object of 24M + 8M stored on the 5th object + 1M
on 6th object 

Maged 

On 2017-10-17 01:59, Christian Wuerdig wrote:

> Maybe an additional example where the numbers don't line up all so
> nicely would be good as well. For example it's not immediately obvious
> to me what would happen with the stripe settings given by your example
> but you write 97M of data
> Would it be 4 objects of 24M and 4 objects of 250KB? Or will the last
> 4 objects be artificially padded (with 0's) to meet the stripe_unit?
> 
> On Tue, Oct 17, 2017 at 12:35 PM, Alexander Kushnirenko
> <kushnirenko at gmail.com> wrote: Hi, Gregory, Ian!
> 
> There is very little information on striper mode in Ceph documentation.
> Could this explanation help?
> 
> The logic of striper mode is very much the same as in RAID-0.  There are 3
> parameters that drives it:
> 
> stripe_unit - the stripe size  (default=4M)
> stripe_count - how many objects to write in parallel (default=1)
> object_size  - when to stop increasing object size and create new objects.
> (default =4M)
> 
> For example if you write 132M of data (132 consecutive pieces of data 1M
> each) in striped mode with the following parameters:
> stripe_unit = 8M
> stripe_count = 4
> object_size = 24M
> Then 8 objects will be created - 4 objects with 24M size and 4 objects with
> 8M size.
> 
> Obj1=24M    Obj2=24M    Obj3=24M    Obj4=24M
> 00 .. 07         08 .. 0f         10 .. 17         18 .. 1f  <-- consecutive
> 1M pieces of data
> 20 .. 27         21 .. 2f         30 .. 37         38 .. 3f
> 40 .. 47         48 .. 4f         50 .. 57         58 .. 5f
> 
> Obj5= 8M    Obj6= 8M    Obj7= 8M    Obj8= 8M
> 60 .. 67        68 .. 6f        70 .. 77        78 .. 7f
> 
> Alexander.
> 
> On Wed, Oct 11, 2017 at 3:19 PM, Alexander Kushnirenko
> <kushnirenko at gmail.com> wrote: 
> Oh!  I put a wrong link, sorry  The picture which explains stripe_unit and
> stripe count is here:
> 
> https://indico.cern.ch/event/330212/contributions/1718786/attachments/642384/883834/CephPluginForXroot.pdf
> 
> I tried to attach it in the mail, but it was blocked.
> 
> On Wed, Oct 11, 2017 at 3:16 PM, Alexander Kushnirenko
> <kushnirenko at gmail.com> wrote: 
> Hi, Ian!
> 
> Thank you for your reference!
> 
> Could you comment on the following rule:
> object_size = stripe_unit * stripe_count
> Or it is not necessarily so?
> 
> I refer to page 8 in this report:
> 
> https://indico.cern.ch/event/531810/contributions/2298934/attachments/1358128/2053937/Ceph-Experience-at-RAL-final.pdf
> 
> Alexander.
> 
> On Wed, Oct 11, 2017 at 1:11 PM, <ian.johnson at stfc.ac.uk> wrote: 
> Hi Gregory
> 
> You're right, when setting the object layout in libradosstriper, one
> should set all three parameters (the number of stripes, the size of the
> stripe unit, and the size of the striped object). The Ceph plugin for
> GridFTP has an example of this at
> https://github.com/stfc/gridFTPCephPlugin/blob/master/ceph_posix.cpp#L371
> 
> At RAL, we use the following values:
> 
> $STRIPER_NUM_STRIPES 1
> 
> $STRIPER_STRIPE_UNIT 8388608
> 
> $STRIPER_OBJECT_SIZE 67108864
> 
> Regards,
> 
> Ian Johnson MBCS
> 
> Data Services Group
> 
> Scientific Computing Department
> 
> Rutherford Appleton Laboratory
> 
> _______________________________________________
> ceph-users mailing list
> ceph-users at lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

_______________________________________________
ceph-users mailing list
ceph-users at lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

 _______________________________________________
ceph-users mailing list
ceph-users at lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ceph.com/pipermail/ceph-users-ceph.com/attachments/20171017/9ce606b8/attachment.html>


More information about the ceph-users mailing list