Re: Error attempting to mount via DFUSE


Pittman, Ashley M
 

 

This specific issue is different appears to be at the dfuse/fuse kernel module level.  I note you’re using a newer fuse driver than I am (7.31 vs 7.23) and also a newer libfuse3 as evidenced by the “Unknown flags 0x3000000” error.  Purely from the output you’ve included the we are not explicitly disabling SPLICE_READ although we should not be using it, however the error from fuse seems to indicate that it’s use is attempted.

 

We do have Ubuntu 20.04 in our CI so I know we test on this, I’ll see if I can find any CI results or if not try Ubuntu 20.04 on a test machine here.

 

Ashley.

 

From: <daos@daos.groups.io> on behalf of Gert Pauwels <gert.pauwels@...>
Reply to: "daos@daos.groups.io" <daos@daos.groups.io>
Date: Monday, 5 October 2020 at 09:09
To: "daos@daos.groups.io" <daos@daos.groups.io>
Subject: Re: [daos] Error attempting to mount via DFUSE

 

I'm experiencing (most likely) the same issue since a week or more on the master branch.

I reproduced it on today's master branch on Ubuntu 20.04. Verified a few days ago on Centos 7.

root@intel-S2600WFD:~/daos-step-by-step# dmg -i pool create --scm-size=30G --nvme-size=300G
Creating DAOS pool with 30 GB SCM and 300 GB NVMe storage (10.00 % ratio)
Pool-create command SUCCEEDED: UUID: fe6475b3-74dc-464f-b8b0-cac50778a9f9, Service replicas: 0

root@intel-S2600WFD:~/daos-step-by-step# daos container create  --svc=0 --path=/mnt/mycontainer --chunk_size=4K --type=POSIX --pool=fe6475b3-74dc-464f-b8b0-cac50778a9f9
fi   INFO src/gurt/fault_inject.c:481 d_fault_inject_init() No config file, fault injection is OFF.
daos INFO src/common/drpc.c:717 drpc_close() Closing dRPC socket fd=3
crt  INFO src/cart/crt_init.c:269 crt_init_opt() libcart version 4.8.0 initializing
crt  WARN src/cart/crt_init.c:161 data_init() FI_UNIVERSE_SIZE was not set; setting to 2048
client INFO src/utils/daos.c:153 cmd_args_print() DAOS system name: daos_server
client INFO src/utils/daos.c:154 cmd_args_print() pool UUID: fe6475b3-74dc-464f-b8b0-cac50778a9f9
client INFO src/utils/daos.c:155 cmd_args_print() cont UUID: 00000000-0000-0000-0000-000000000000
client INFO src/utils/daos.c:157 cmd_args_print() pool svc: parsed 1 ranks from input 0
client INFO src/utils/daos.c:161 cmd_args_print() attr: name=NULL, value=NULL
client INFO src/utils/daos.c:165 cmd_args_print() path=/mnt/mycontainer, type=POSIX, oclass=UNKNOWN, chunk_size=4096
client INFO src/utils/daos.c:168 cmd_args_print() snapshot: name=NULL, epoch=0, epoch range=NULL (0-0)
client INFO src/utils/daos.c:174 cmd_args_print() oid: 0.0
daos INFO src/common/drpc.c:717 drpc_close() Closing dRPC socket fd=18
daos INFO src/common/drpc.c:717 drpc_close() Closing dRPC socket fd=18
Successfully created container 0912dece-00e1-4e8f-8d7f-ac63603f52ac type POSIX

root@intel-S2600WFD:~/daos-step-by-step# dfuse --pool=fe6475b3-74dc-464f-b8b0-cac50778a9f9 --cont=0912dece-00e1-4e8f-8d7f-ac63603f52ac --mountpoint=/mnt/1 --svc=0 --foreground
fi   INFO src/gurt/fault_inject.c:481 d_fault_inject_init() No config file, fault injection is OFF.
daos INFO src/common/drpc.c:717 drpc_close() Closing dRPC socket fd=3
crt  INFO src/cart/crt_init.c:269 crt_init_opt() libcart version 4.8.0 initializing
crt  WARN src/cart/crt_init.c:161 data_init() FI_UNIVERSE_SIZE was not set; setting to 2048
duns INFO src/client/dfs/duns.c:393 duns_resolve_path() Path does not represent a DAOS link
dfuse INFO src/client/dfuse/dfuse_main.c:436 main(0x55639bbfb100) duns_resolve_path() returned 61 No data available
daos INFO src/common/drpc.c:717 drpc_close() Closing dRPC socket fd=18
daos INFO src/common/drpc.c:717 drpc_close() Closing dRPC socket fd=18
dfuse INFO src/client/dfuse/dfuse_fuseops.c:72 dfuse_fuse_init(0x55639bc6eb30) Fuse configuration
dfuse INFO src/client/dfuse/dfuse_fuseops.c:74 dfuse_fuse_init(0x55639bc6eb30) Proto 7 31
dfuse INFO src/client/dfuse/dfuse_fuseops.c:84 dfuse_fuse_init(0x55639bc6eb30) max read 0x400000
dfuse INFO src/client/dfuse/dfuse_fuseops.c:85 dfuse_fuse_init(0x55639bc6eb30) max write 0x400000
dfuse INFO src/client/dfuse/dfuse_fuseops.c:86 dfuse_fuse_init(0x55639bc6eb30) readahead 0x20000
dfuse INFO src/client/dfuse/dfuse_fuseops.c:88 dfuse_fuse_init(0x55639bc6eb30) Capability supported 0x31fffdb
dfuse INFO src/client/dfuse/dfuse_fuseops.c:39 dfuse_show_flags(0x55639bc6eb30) Flag FUSE_CAP_ASYNC_READ enabled
dfuse INFO src/client/dfuse/dfuse_fuseops.c:40 dfuse_show_flags(0x55639bc6eb30) Flag FUSE_CAP_POSIX_LOCKS enabled
dfuse INFO src/client/dfuse/dfuse_fuseops.c:41 dfuse_show_flags(0x55639bc6eb30) Flag FUSE_CAP_ATOMIC_O_TRUNC enabled
dfuse INFO src/client/dfuse/dfuse_fuseops.c:42 dfuse_show_flags(0x55639bc6eb30) Flag FUSE_CAP_EXPORT_SUPPORT enabled
dfuse INFO src/client/dfuse/dfuse_fuseops.c:43 dfuse_show_flags(0x55639bc6eb30) Flag FUSE_CAP_DONT_MASK enabled
dfuse INFO src/client/dfuse/dfuse_fuseops.c:44 dfuse_show_flags(0x55639bc6eb30) Flag FUSE_CAP_SPLICE_WRITE enabled
dfuse INFO src/client/dfuse/dfuse_fuseops.c:45 dfuse_show_flags(0x55639bc6eb30) Flag FUSE_CAP_SPLICE_MOVE enabled
dfuse INFO src/client/dfuse/dfuse_fuseops.c:46 dfuse_show_flags(0x55639bc6eb30) Flag FUSE_CAP_SPLICE_READ enabled
dfuse INFO src/client/dfuse/dfuse_fuseops.c:47 dfuse_show_flags(0x55639bc6eb30) Flag FUSE_CAP_FLOCK_LOCKS enabled
dfuse INFO src/client/dfuse/dfuse_fuseops.c:48 dfuse_show_flags(0x55639bc6eb30) Flag FUSE_CAP_IOCTL_DIR enabled
dfuse INFO src/client/dfuse/dfuse_fuseops.c:49 dfuse_show_flags(0x55639bc6eb30) Flag FUSE_CAP_AUTO_INVAL_DATA enabled
dfuse INFO src/client/dfuse/dfuse_fuseops.c:50 dfuse_show_flags(0x55639bc6eb30) Flag FUSE_CAP_READDIRPLUS enabled
dfuse INFO src/client/dfuse/dfuse_fuseops.c:51 dfuse_show_flags(0x55639bc6eb30) Flag FUSE_CAP_READDIRPLUS_AUTO enabled
dfuse INFO src/client/dfuse/dfuse_fuseops.c:52 dfuse_show_flags(0x55639bc6eb30) Flag FUSE_CAP_ASYNC_DIO enabled
dfuse INFO src/client/dfuse/dfuse_fuseops.c:53 dfuse_show_flags(0x55639bc6eb30) Flag FUSE_CAP_WRITEBACK_CACHE enabled
dfuse INFO src/client/dfuse/dfuse_fuseops.c:54 dfuse_show_flags(0x55639bc6eb30) Flag FUSE_CAP_NO_OPEN_SUPPORT enabled
dfuse INFO src/client/dfuse/dfuse_fuseops.c:55 dfuse_show_flags(0x55639bc6eb30) Flag FUSE_CAP_PARALLEL_DIROPS enabled
dfuse INFO src/client/dfuse/dfuse_fuseops.c:56 dfuse_show_flags(0x55639bc6eb30) Flag FUSE_CAP_POSIX_ACL enabled
dfuse INFO src/client/dfuse/dfuse_fuseops.c:57 dfuse_show_flags(0x55639bc6eb30) Flag FUSE_CAP_HANDLE_KILLPRIV enabled
dfuse ERR  src/client/dfuse/dfuse_fuseops.c:60 dfuse_show_flags(0x55639bc6eb30) Unknown flags 0x3000000
dfuse INFO src/client/dfuse/dfuse_fuseops.c:92 dfuse_fuse_init(0x55639bc6eb30) Capability requested 0x149a09
dfuse INFO src/client/dfuse/dfuse_fuseops.c:39 dfuse_show_flags(0x55639bc6eb30) Flag FUSE_CAP_ASYNC_READ enabled
dfuse INFO src/client/dfuse/dfuse_fuseops.c:40 dfuse_show_flags(0x55639bc6eb30) Flag FUSE_CAP_POSIX_LOCKS disabled
dfuse INFO src/client/dfuse/dfuse_fuseops.c:41 dfuse_show_flags(0x55639bc6eb30) Flag FUSE_CAP_ATOMIC_O_TRUNC enabled
dfuse INFO src/client/dfuse/dfuse_fuseops.c:42 dfuse_show_flags(0x55639bc6eb30) Flag FUSE_CAP_EXPORT_SUPPORT disabled
dfuse INFO src/client/dfuse/dfuse_fuseops.c:43 dfuse_show_flags(0x55639bc6eb30) Flag FUSE_CAP_DONT_MASK disabled
dfuse INFO src/client/dfuse/dfuse_fuseops.c:44 dfuse_show_flags(0x55639bc6eb30) Flag FUSE_CAP_SPLICE_WRITE disabled
dfuse INFO src/client/dfuse/dfuse_fuseops.c:45 dfuse_show_flags(0x55639bc6eb30) Flag FUSE_CAP_SPLICE_MOVE disabled
dfuse INFO src/client/dfuse/dfuse_fuseops.c:46 dfuse_show_flags(0x55639bc6eb30) Flag FUSE_CAP_SPLICE_READ enabled
dfuse INFO src/client/dfuse/dfuse_fuseops.c:47 dfuse_show_flags(0x55639bc6eb30) Flag FUSE_CAP_FLOCK_LOCKS disabled
dfuse INFO src/client/dfuse/dfuse_fuseops.c:48 dfuse_show_flags(0x55639bc6eb30) Flag FUSE_CAP_IOCTL_DIR enabled
dfuse INFO src/client/dfuse/dfuse_fuseops.c:49 dfuse_show_flags(0x55639bc6eb30) Flag FUSE_CAP_AUTO_INVAL_DATA enabled
dfuse INFO src/client/dfuse/dfuse_fuseops.c:50 dfuse_show_flags(0x55639bc6eb30) Flag FUSE_CAP_READDIRPLUS disabled
dfuse INFO src/client/dfuse/dfuse_fuseops.c:51 dfuse_show_flags(0x55639bc6eb30) Flag FUSE_CAP_READDIRPLUS_AUTO disabled
dfuse INFO src/client/dfuse/dfuse_fuseops.c:52 dfuse_show_flags(0x55639bc6eb30) Flag FUSE_CAP_ASYNC_DIO enabled
dfuse INFO src/client/dfuse/dfuse_fuseops.c:53 dfuse_show_flags(0x55639bc6eb30) Flag FUSE_CAP_WRITEBACK_CACHE disabled
dfuse INFO src/client/dfuse/dfuse_fuseops.c:54 dfuse_show_flags(0x55639bc6eb30) Flag FUSE_CAP_NO_OPEN_SUPPORT disabled
dfuse INFO src/client/dfuse/dfuse_fuseops.c:55 dfuse_show_flags(0x55639bc6eb30) Flag FUSE_CAP_PARALLEL_DIROPS enabled
dfuse INFO src/client/dfuse/dfuse_fuseops.c:56 dfuse_show_flags(0x55639bc6eb30) Flag FUSE_CAP_POSIX_ACL disabled
dfuse INFO src/client/dfuse/dfuse_fuseops.c:57 dfuse_show_flags(0x55639bc6eb30) Flag FUSE_CAP_HANDLE_KILLPRIV enabled
dfuse INFO src/client/dfuse/dfuse_fuseops.c:99 dfuse_fuse_init(0x55639bc6eb30) max_background 16
dfuse INFO src/client/dfuse/dfuse_fuseops.c:100 dfuse_fuse_init(0x55639bc6eb30) congestion_threshold 8
fuse: splice from device: Invalid argument
dfuse ERR  src/client/dfuse/dfuse_main.c:191 ll_loop_fn(0x55639bbfb100) Fuse loop exited with return code: -22
dfuse ERR  src/client/dfuse/dfuse_core.c:384 dfuse_start(0x55639bc6eb30) Unable to register FUSE fs
dfuse ERR  src/client/dfuse/dfuse_core.c:399 dfuse_start(0x55639bc6eb30) Failed to start dfuse, rc: -1003
dfuse ERR  src/client/dfuse/dfuse_main.c:519 main(0x55639bc8c2f0) DFP left at the end
dfuse ERR  src/client/dfuse/dfuse_main.c:522 main(0x55639bc8c3f0) DFS left at the end
dfuse INFO src/client/dfuse/dfuse_main.c:561 main() Exiting with status -1003


When --pool and --cont ar not specified and --mountpoint pointing to the path specified when creating the container, you get the same output as above.
Calling dfuse --mountpoint=/mnt/mycontainer --svc=0 --foreground gives the same output as as dfuse --pool=fe6475b3-74dc-464f-b8b0-cac50778a9f9 --cont=0912dece-00e1-4e8f-8d7f-ac63603f52ac --mountpoint=/mnt/1 --svc=0 --foreground

Gert,

---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

Join daos@daos.groups.io to automatically receive all group messages.