[ceph-users] bucket cloning/writable snapshots

Haomai Wang haomai at xsky.com
Sat Nov 18 00:51:50 PST 2017


On Sat, Nov 18, 2017 at 4:49 PM, Fred Gansevles <fred at betterbe.com> wrote:

> Hi,
>
> Currently our company has +/- 50 apps where every app has its own
> data-area on NFS.
> We need to switch S3, using Ceph, as our new data layer with
> every app using its own s3-bucket, equivalent to the NFS data-area.
> The sizes of the data-areas, depending on the app, varies from 1.3 GB to
> 358 GB.
>
> In order to test multiple versions of the app, we currently make a
> writable snapshot of
> the data-area to avoid copying or 'polluting' the original (i.e.
> 'production' data).
> Since a snapshot is fast we can make multiple snapshots easy and discard
> them
> afterwards.
>
> With Ceph, we would like to do something alike, i.e. 'fast' copying and
> easily discardable.
> The path we are trying to take is: make a 'clone' of the bucket (i.e.
> writable
> snapshots) into a test-bucket.
>

why not S3 multiversion?

>
> Our design for this is the following:
> - We need to have every app bucket in its own unique pool-set.
> - (on the ceph-nodes) determine the pool-set that is used by the given
> bucket
> - (on the ceph-nodes) make snapshots of the pools and assign these
> snapshots to
>   an newly created (writable!) test-bucket.
> - after the test is finished, the test-bucket can be removed (either from
> the ceph-node
>   or the test system).
>
> The test-procedure that is testing the app is aware of its environment,
> i.e.: it
> 'knows' that it has to do specials things to get a test-bucket. The app
> itself isn not
> aware of this, and just uses whatever bucket is passed to it.
>
> This way the test procedure can use the test-bucket and does not interfere
> with the
> original data and the app can be run 'as-is' without changing the code.
>
> I have the following question:
>     Is this scenario at all possible?
>     - if yes: how can I accomplish this?
>     - if no: is this a design-flaw (it can be done, just not this way)
>              or its simply not possible.
>
> --
>
> Best regards,
>
> Fred Gansevles
> *Devops Engineer*
> <https://www.betterbe.com>
> Auke Vleerstraat 140 E *T* +31 (0) 53 48 00 680
> <+31%20%280%29%2053%2048%2000%20680>
> 7547 AN Enschede *E* fgansevles at betterbe.com
> CC no. 08097527
> <https://www.kvk.nl/orderstraat/product-kiezen/?kvknummer=080975270000>
> *M* +31 (0)6 30 262 174 <+31%20%280%296%2030%20262%20174> www.betterbe.com
> BetterBe accepts no liability for the content of this email, or for the
> consequences of any actions taken on the basis of the information provided,
> unless that information is subsequently confirmed in writing. If you are
> not the intended recipient you are notified that disclosing, copying,
> distributing or taking any action in reliance on the contents of this
> information is strictly prohibited.
>
> _______________________________________________
> ceph-users mailing list
> ceph-users at lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ceph.com/pipermail/ceph-users-ceph.com/attachments/20171118/69827502/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: acaljfbihadinhoa.png
Type: image/png
Size: 10832 bytes
Desc: not available
URL: <http://lists.ceph.com/pipermail/ceph-users-ceph.com/attachments/20171118/69827502/attachment.png>


More information about the ceph-users mailing list