[ceph-users] who is using nfs-ganesha and cephfs?

Supriti Singh Supriti.Singh at suse.com
Thu Nov 9 03:15:24 PST 2017


Hi Sage,

As Lars mentioned, at SUSE, we use ganesha 2.5.2/luminous. We did a preliminary performance comparison of cephfs client
and nfs-ganesha client. I have attached the results. The results are aggregate bandwidth over 10 clients.

1. Test Setup:
We use fio to read/write to a single 5GB file per thread for 300 seconds. A single job (represented in x-axis) is of
type {number_of_worker_thread}rw_{block_size}_{op}, where, 
number_of_worker_threads: 1, 4, 8, 16
Block size: 4K,64K,1M,4M,8M
op: rw 

 
2. NFS-Ganesha configuration:
Parameters set (other than default):
1. Graceless = True
2. MaxRPCSendBufferSize/MaxRPCRecvBufferSize is set to max value.

3. Observations:
-  For single thread (on each client) and 4k block size, the b/w is around 45% of cephfs 
- As number of threads increases, the performance drops. It could be related to nfs-ganesha parameter
"Dispatch_Max_Reqs_Xprt", which defaults to 512. Note, this parameter is important only for v2.5. 
- We did run with both nfs-ganesha mdcache enabled/disabled. But there were no significant improvements with caching.
Not sure but it could be related to this issue: https://github.com/nfs-ganesha/nfs-ganesha/issues/223
  
The results are still preliminary, and I guess with proper tuning of nfs-ganesha parameters, it could be better.

Thanks,
Supriti 

------
Supriti Singh SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton,
HRB 21284 (AG Nürnberg)
 



>>> Lars Marowsky-Bree <lmb at suse.com> 11/09/17 11:07 AM >>>
On 2017-11-08T21:41:41, Sage Weil <sweil at redhat.com> wrote:

> Who is running nfs-ganesha's FSAL to export CephFS?  What has your 
> experience been?
> 
> (We are working on building proper testing and support for this into 
> Mimic, but the ganesha FSAL has been around for years.)

We use it currently, and it works, but let's not discuss the performance
;-)

How else do you want to build this into Mimic?

Regards,
    Lars

-- 
SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)
"Experience is the name everyone gives to their mistakes." -- Oscar Wilde

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo at vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ceph.com/pipermail/ceph-users-ceph.com/attachments/20171109/b8ccb65f/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: NFS_Ganesha_vs_CephFS.ods
Type: application/octet-stream
Size: 25399 bytes
Desc: not available
URL: <http://lists.ceph.com/pipermail/ceph-users-ceph.com/attachments/20171109/b8ccb65f/attachment.obj>


More information about the ceph-users mailing list