[ceph-users] RGW Swift metadata dropped when S3 bucket versioning enabled

Maxime Guyot maxime at root314.com
Wed Nov 28 10:06:39 PST 2018


Hi Florian,

You assumed correctly, the "test" container (private) was created with the
"openstack container create test", then I am using the S3 API to
enable/disable object versioning on it.
I use the following Python snippet to enable/disable S3 bucket versioning:

import boto, boto.s3, boto.s3.connection
conn = conn = boto.connect_s3(aws_access_key_id='***',
aws_secret_access_key='***', host='***', port=8080,
calling_format=boto.s3.connection.OrdinaryCallingFormat())
bucket = conn.get_bucket('test')
bucket.configure_versioning(True) # Or False to disable S3 bucket versioning
bucket.get_versioning_status()

> Semi-related: I've seen some interesting things when mucking around with
> a single container/bucket while switching APIs, when it comes to
> container properties and metadata. For example, if you set a public read
> ACL on an S3 bucket, the the corresponding Swift container is also
> publicly readable but its read ACL looks empty (i.e. private) when you
> ask via the Swift API.

This can definitely become a problem if Swift API says "private" but data
is actually publicly available.
Since the doc says "S3 and Swift APIs share a common namespace, so you may
write data with one API and retrieve it with the other", it might be useful
to document this kind of limitations somewhere.

Cheers,
/ Maxime

On Wed, 28 Nov 2018 at 17:58 Florian Haas <florian at citynetwork.eu> wrote:

> On 27/11/2018 20:28, Maxime Guyot wrote:
> > Hi,
> >
> > I'm running into an issue with the RadosGW Swift API when the S3 bucket
> > versioning is enabled. It looks like it silently drops any metadata sent
> > with the "X-Object-Meta-foo" header (see example below).
> > This is observed on a Luminous 12.2.8 cluster. Is that a normal thing?
> > Am I misconfiguring something here?
> >
> >
> > With S3 bucket versioning OFF:
> > $ openstack object set --property foo=bar test test.dat
> > $ os object show test test.dat
> > +----------------+----------------------------------+
> > | Field          | Value                            |
> > +----------------+----------------------------------+
> > | account        | v1                               |
> > | container      | test                             |
> > | content-length | 507904                           |
> > | content-type   | binary/octet-stream              |
> > | etag           | 03e8a398f343ade4e1e1d7c81a66e400 |
> > | last-modified  | Tue, 27 Nov 2018 13:53:54 GMT    |
> > | object         | test.dat                         |
> > | properties     | Foo='bar'                        |  <= Metadata is
> here
> > +----------------+----------------------------------+
> >
> > With S3 bucket versioning ON:
>
> Can you elaborate on what exactly you're doing here to enable S3 bucket
> versioning? Do I assume correctly that you are creating the "test"
> container using the swift or openstack client, then sending a
> VersioningConfiguration request against the "test" bucket, as explained
> in
>
> https://docs.aws.amazon.com/AmazonS3/latest/dev/Versioning.html#how-to-enable-disable-versioning-intro
> ?
>
> > $ openstack object set --property foo=bar test test2.dat
> > $ openstack object show test test2.dat
> > +----------------+----------------------------------+
> > | Field          | Value                            |
> > +----------------+----------------------------------+
> > | account        | v1                               |
> > | container      | test                             |
> > | content-length | 507904                           |
> > | content-type   | binary/octet-stream              |
> > | etag           | 03e8a398f343ade4e1e1d7c81a66e400 |
> > | last-modified  | Tue, 27 Nov 2018 13:56:50 GMT    |
> > | object         | test2.dat                        | <= Metadata is
> absent
> > +----------------+----------------------------------+
>
> Semi-related: I've seen some interesting things when mucking around with
> a single container/bucket while switching APIs, when it comes to
> container properties and metadata. For example, if you set a public read
> ACL on an S3 bucket, the the corresponding Swift container is also
> publicly readable but its read ACL looks empty (i.e. private) when you
> ask via the Swift API.
>
> Cheers,
> Florian
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ceph.com/pipermail/ceph-users-ceph.com/attachments/20181128/6b84202c/attachment.html>


More information about the ceph-users mailing list