[ceph-users] who is using nfs-ganesha and cephfs?

Supriti Singh Supriti.Singh at suse.com
Thu Nov 9 03:15:24 PST 2017

Hi Sage,

As Lars mentioned, at SUSE, we use ganesha 2.5.2/luminous. We did a preliminary performance comparison of cephfs client
and nfs-ganesha client. I have attached the results. The results are aggregate bandwidth over 10 clients.

1. Test Setup:
We use fio to read/write to a single 5GB file per thread for 300 seconds. A single job (represented in x-axis) is of
type {number_of_worker_thread}rw_{block_size}_{op}, where, 
number_of_worker_threads: 1, 4, 8, 16
Block size: 4K,64K,1M,4M,8M
op: rw 

2. NFS-Ganesha configuration:
Parameters set (other than default):
1. Graceless = True
2. MaxRPCSendBufferSize/MaxRPCRecvBufferSize is set to max value.

3. Observations:
-  For single thread (on each client) and 4k block size, the b/w is around 45% of cephfs 
- As number of threads increases, the performance drops. It could be related to nfs-ganesha parameter
"Dispatch_Max_Reqs_Xprt", which defaults to 512. Note, this parameter is important only for v2.5. 
- We did run with both nfs-ganesha mdcache enabled/disabled. But there were no significant improvements with caching.
Not sure but it could be related to this issue: https://github.com/nfs-ganesha/nfs-ganesha/issues/223
The results are still preliminary, and I guess with proper tuning of nfs-ganesha parameters, it could be better.


Supriti Singh SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton,
HRB 21284 (AG Nürnberg)

>>> Lars Marowsky-Bree <lmb at suse.com> 11/09/17 11:07 AM >>>
On 2017-11-08T21:41:41, Sage Weil <sweil at redhat.com> wrote:

> Who is running nfs-ganesha's FSAL to export CephFS?  What has your 
> experience been?
> (We are working on building proper testing and support for this into 
> Mimic, but the ganesha FSAL has been around for years.)

We use it currently, and it works, but let's not discuss the performance

How else do you want to build this into Mimic?


SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)
"Experience is the name everyone gives to their mistakes." -- Oscar Wilde

To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo at vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ceph.com/pipermail/ceph-users-ceph.com/attachments/20171109/b8ccb65f/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: NFS_Ganesha_vs_CephFS.ods
Type: application/octet-stream
Size: 25399 bytes
Desc: not available
URL: <http://lists.ceph.com/pipermail/ceph-users-ceph.com/attachments/20171109/b8ccb65f/attachment.obj>

More information about the ceph-users mailing list