[ceph-users] S3 object-size based storage placement policy

David Watzke watzke at avast.com
Wed Nov 1 09:05:49 PDT 2017

Hello everyone,

I would like to implement an object-size based pool-placement policy to 
be used with the S3 API and I'd like to ask for some insights.

In particular, I would like to automatically store objects (via S3 API) 
into different pools based on their size. I.e. <64K objects to a SSD 
pool with replication factor 3 and any larger objects to HDD-based pool 
with erasure coding 4+2.

I would like to do this seamlessly for (S3 API) users - small objects 
automatically go to the quick storage, large objects go elsewhere. Then 
when you fetch the objects, you also shouldn't need to worry about this. 
I don't mind using some external tools to achieve this (nginx reverse 
proxy in front of the object gateway) if needed, as long as the approach 
is reliable.

I found this howto describing how to put objects using a custom storage 
policy via the Swift API:


I was looking for a similiar S3 example and only stumbled upon this 
(couldn't find any other relevant info):


It says that an S3 bucket has to be created with a location constraint 
so I guess it's not possible to specify the storage policy per-object by 
sending a header just like in Swift API? (which would be easily 
achievable with a reverse proxy in front of the object gateway)

Is there some other way to achieve my goal?

Thanks in advance,

David Watzke

More information about the ceph-users mailing list