Hi Anton, The last one that we’ve tried and worked for us was MOFED 5.0.2 which is what we currently use for our test clusters that use ofi+verbs;ofi_rxm provider. Thanks, ~~Alex.

Hi Tom, After booting I run the following 2 commands: root@intel-S2600WFD:~/daos# cat /proc/meminfo | grep Huge AnonHugePages: 0 kB ShmemHugePages: 0 kB FileHugePages: 0 kB

MOFED 5.0 works well, 5.1 seems to have an incompatibility with libfabric currently. (Note this is not a supported list or anything - This is just what we've used successfully.) Regards, -Patrick

Hello Gert, It’s likely the thread lib init failure is due to a problem with hugepages configuration as spdk_thread_lib_init() only calls spdk_mempool_create(). Can you please do some debug

I'm using Optane PMEM ( there are 4 modules 512GB each, two PMEM modules near each socket). I created /dev/pmem0 and /dev/pmem1 devices using `ipmctl create -goal PersistentMemoryType=AppDirect`

It sounds like the DAOS server is not able to register memory to initiate a RDMA. Could you please tell me more about the network and storage you use on the server? Optane PMEM or DRAM? Also, what

This is errors from daos_server.log : 09/11-20:41:22.58 apache512 DAOS[10757/10786] hg ERR # NA -- Error -- /builddir/build/BUILD/mercury-2.0.0a1/src/na/na_ofi.c:4196 # na_ofi_mem_register():

Hi Tom, After applying the patch you provided the error is now: 09/11-17:47:17.20 intel-S2600WFD DAOS[11713/11713] bio ERR src/bio/bio_xst 09/11-17 ream.c:369 bio_spdk_env_init() Failed to

Thanks Gurt, Yes, that was only to get the identify application running, which looks like it runs fine as expected given the problem looks like initializing the thread library which identify

Hello, You and Gurt are experiencing the same issue: 09/10-14:23:55.96 intel-S2600WFD DAOS[260088/260088] bio ERR src/bio/bio_xstream.c:367 bio_spdk_env_init() Failed to init SPDK thread

Hi Tom, I set the variable LD_LIBRARY_PATH as you indicated to spdk library path and that indeed solves the problem for the identify application. Now the identify application returns all the data as

Johann thanks! You right, and also I've run daos_test compiled from source. Now I've ran from package daos-tests (RPM of v1.0.1) and it works, but one test is stuck: DAOS_IOD_SINGLE:NVMe size:

Looks better. The network stack seems to be properly initialized now. The failure you see now (“undefined symbol: dmg_pool_create”) is related to the C wrapper of the dmg command which is used to

Hi, we run 1.0.1 with these settings: # DAOS 1.0.1 client-side settings for Mellanox InfiniBand: export OFI_INTERFACE="ib0" export CRT_PHY_ADDR_STR="ofi+verbs;ofi_rxm" export

Could you please try with export OFI_DOMAIN="mlx5_0"? If it does not work, please collect the the daos logs (i.e. /tmp/daos.log). Cheers, Johann From:<daos@daos.groups.io> on behalf of

Johann thanks! I've set export CRT_PHY_ADDR_STR="ofi+verbs;ofi_rxm" both on storage server and client and now I'get an error -1020: export OFI_INTERFACE=ib0 export

Hi Anton, Since you use 1.0.1, you also need to set CRT_PHY_ADDR_STR (not required any longer in 1.2/master) as follows: export CRT_PHY_ADDR_STR="ofi+verbs;ofi_rxm" Hope this helps.

Hi everyone! Im trying to make simple cluster of two nodes connected to IB network. One node is with PMEM and other one is client (for daos agent). I installed rpm for centos 7 v1.0.1 Here is

Same problem here, I used the master branch of daos on ubuntu 20.04.1 I also met this error when starting the daos_server:The same message was printed in /tmp/daos_server.log: The identify error also

Hi Johann, Thank you for your reply. It helps a lot. Regards, Changwoo Min