[ceph-users] [Ceph-cn] librados: Objecter returned from getxattrs r=-2

Zhou, Yuan yuan.zhou at intel.com
Mon Nov 2 23:39:45 PST 2015


Hi,

So in RGW there's no hive* objects now, could you please check if there's any exists in the S3 perspective?  That's to check the object listing of bucket 'olla' via the S3 API (boto or s3cmd could do the job)

I've met some similar issue in Hadoop over SwiftFS before. There's some OSDs were down in Ceph cluster, then the file listing in Hadoop and Swift does not match. Don’t know the detail failures though. I was simply trying to do some benchmarks so the data are not important. By manually deleting the objects/buckets and regenerating the data issue was fixed.

hope this can help. 

thanks, -yuan

-----Original Message-----
From: 张绍文 [mailto:zhangshaowen at btte.net] 
Sent: Tuesday, November 3, 2015 1:45 PM
To: Zhou, Yuan
Cc: ceph-cn at lists.ceph.com; ceph-users at ceph.com
Subject: Re: [Ceph-cn] librados: Objecter returned from getxattrs r=-2

On Tue, 3 Nov 2015 05:32:27 +0000
"Zhou, Yuan" <yuan.zhou at intel.com> wrote:

> Hi,
> 
> The directory there should be some simulated hierarchical structure 
> with '/' in the object names. Do you mind checking the rest objects in 
> ceph pool .rgw.buckets?
> 
> $ rados ls -p .rgw.buckets | grep default.157931.5_hive
> 
> If there're still objects come out, you might try to delete them from 
> the 'olla' bucket with S3 API. (Note I'm not sure how's your Hive data 
> generated, so please do backup first if it's important.)
> 

Thanks for your reply. I dumped object list yesterday:

# rados -p .rgw.buckets ls >obj-list
# ls -lh obj-list
-rw-r--r-- 1 root root 1.2G Nov  2 15:51 obj-list # grep default.157931.5_hive obj-list # 

There's no such object.

> 
> -----Original Message-----
> From: Ceph-cn [mailto:ceph-cn-bounces at lists.ceph.com] On Behalf Of ???
> Sent: Tuesday, November 3, 2015 12:22 PM
> To: ceph-cn at lists.ceph.com; ceph-users at ceph.com
> Subject: Re: [Ceph-cn] librados: Objecter returned from getxattrs r=-2
> 
> With debug_objecter = 20/0 I get this, I guess the thing is: the 
> object has been removed, but "directory" info still exists.
> 
> 2015-11-03 12:07:22.264704 7f03c42f3700 10 client.214496.objecter 
> ms_dispatch 0x2c18840 osd_op_reply(81 
> default.157931.5_hive/staging_hive_2015-11-01_14-57-40_861_37977797652
> 10222008-1/_tmp.-ext-10000/ [getxattrs,stat] v0'0 uv0 ack = -2 ((2) No 
> such file or directory)) v6
> 
> So, how can I safely remove the "directory" info?
> 
> On Tue, 3 Nov 2015 10:10:26 +0800
> 张绍文 <zhangshaowen at btte.net> wrote:
> 
> > On Mon, 2 Nov 2015 16:47:11 +0800
> > 张绍文 <zhangshaowen at btte.net> wrote:
> >   
> > > On Mon, 2 Nov 2015 16:36:57 +0800
> > > 张绍文 <zhangshaowen at btte.net> wrote:
> > >     
> > > > Hi, all:
> > > > 
> > > > I'm using hive via s3a, but it's not usable after I removed some 
> > > > temp files with:
> > > > 
> > > > /opt/hadoop/bin/hdfs dfs -rm -r -f s3a://olla/hive/
> > > > 
> > > > With debug_radosgw = 10/0, I got these messages repeatly:
> > > > 
> > > > 2015-11-02 14:30:44.547271 7f08ef7fe700 10 librados: Objecter 
> > > > returned from getxattrs r=-2 2015-11-02 14:30:44.549117
> > > > 7f08ef7fe700 10 librados: getxattrs
> > > > oid=default.157931.5_hive/staging_hive_2015-11-01_14-57-40_861_3
> > > > 79 7779765210222008-1/_tmp.-ext-10000/
> > > > nspace=
> > > > 
> > > > I dumped whole object list, and there's no object named starts 
> > > > with hive/..., and hive is not usable now, please help.
> > > >       
> > > 
> > > Sorry, I forgot this:
> > > 
> > > # ceph -v
> > > ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3)
> > > 
> > > Other known "directories" under THE same bucket is readable.
> > > 
> > >     
> > 
> > Also happened to others on ceph-users maillist, seems not resolved:
> > 
> > http://article.gmane.org/gmane.comp.file-systems.ceph.user/7653/matc
> > h=
> > objecter+returned+getxattrs
> > 
> >   
> 
> 
> 
> --
> 张绍文
> _______________________________________________
> Ceph-cn mailing list
> Ceph-cn at lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-cn-ceph.com



--
张绍文


More information about the ceph-users mailing list