Re: DAOS_test failed


Lombardi, Johann
 

It sounds like the DAOS server is not able to register memory to initiate a RDMA. Could you please tell me more about the network and storage you use on the server? Optane PMEM or DRAM? Also, what version of OFED do you use?

 

Cheers,

Johann

 

From: <daos@daos.groups.io> on behalf of "anton.brekhov@..." <anton.brekhov@...>
Reply-To: "daos@daos.groups.io" <daos@daos.groups.io>
Date: Friday 11 September 2020 at 19:41
To: "daos@daos.groups.io" <daos@daos.groups.io>
Subject: Re: [daos] DAOS_test failed

 

This is errors from daos_server.log :

09/11-20:41:22.58 apache512 DAOS[10757/10786] hg   ERR  # NA -- Error -- /builddir/build/BUILD/mercury-2.0.0a1/src/na/na_ofi.c:4196

 # na_ofi_mem_register(): fi_mr_reg() failed, rc: -95 (Operation not supported)

09/11-20:41:26.20 apache512 DAOS[10757/10786] hg   ERR  # HG -- Error -- /builddir/build/BUILD/mercury-2.0.0a1/src/mercury_bulk.c:494

 # hg_bulk_create(): NA_Mem_register() failed (NA_PROTOCOL_ERROR)

09/11-20:41:26.20 apache512 DAOS[10757/10786] hg   ERR  # HG -- Error -- /builddir/build/BUILD/mercury-2.0.0a1/src/mercury_bulk.c:1072

 # HG_Bulk_create(): Could not create bulk handle

09/11-20:41:26.20 apache512 DAOS[10757/10786] hg   ERR  src/cart/crt_hg.c:1445 crt_hg_bulk_create() HG_Bulk_create failed, hg_ret: 11.

09/11-20:41:26.20 apache512 DAOS[10757/10786] bulk ERR  src/cart/crt_bulk.c:137 crt_bulk_create() crt_hg_bulk_create failed, rc: -1020.

09/11-20:41:26.20 apache512 DAOS[10757/10786] object ERR  src/object/srv_obj.c:359 obj_bulk_transfer() crt_bulk_create 0 error (-1020).

09/11-20:41:26.20 apache512 DAOS[10757/10786] object ERR  src/object/srv_obj.c:992 obj_local_rw() 1155473290706288642.0.1 data transfer failed, dma 1 rc DER_HG(-1020)

09/11-20:41:26.20 apache512 DAOS[10757/10786] object ERR  src/object/srv_obj.c:96 obj_rw_complete() 1155473290706288642.0.1Fetch end failed: -1020

09/11-20:41:26.20 apache512 DAOS[10757/10777] rpc  DBUG src/cart/crt_register.c:215 crt_opc_lookup() looking up opcode: 0x2010003

09/11-20:41:26.20 apache512 DAOS[10757/10777] bio  DBUG src/bio/bio_buffer.c:824 copy_one() bio copy 0x7f8c98212380 size 24

09/11-20:41:26.20 apache512 DAOS[10757/10777] bio  DBUG src/bio/bio_buffer.c:824 copy_one() bio copy 0x7f8c981e3680 size 320

09/11-20:41:26.20 apache512 DAOS[10757/10777] vos  DBUG src/vos/vos_io.c:607 akey_fetch() akey [16] fetch single epr 9-6359751

09/11-20:41:26.20 apache512 DAOS[10757/10777] bio  DBUG src/bio/bio_buffer.c:824 copy_one() bio copy 0x7f8c98207700 size 8

09/11-20:41:26.20 apache512 DAOS[10757/10777] vos  DBUG src/vos/vos_io.c:607 akey_fetch() akey [12] fetch single epr 5-6359751

09/11-20:41:26.20 apache512 DAOS[10757/10777] bio  DBUG src/bio/bio_buffer.c:824 copy_one() bio copy 0x7f8c98241400 size 4

09/11-20:41:26.20 apache512 DAOS[10757/10777] vos  DBUG src/vos/vos_io.c:607 akey_fetch() akey [11] fetch single epr 5-6359751

09/11-20:41:26.20 apache512 DAOS[10757/10786] rpc  DBUG src/cart/crt_register.c:215 crt_opc_lookup() looking up opcode: 0x4010001

09/11-20:41:26.20 apache512 DAOS[10757/10786] object DBUG src/object/srv_obj.c:1337 ds_obj_rw_handler() overwrite epoch 1599846087457767431

09/11-20:41:26.20 apache512 DAOS[10757/10786] vos  DBUG src/vos/vos_io.c:607 akey_fetch() akey [1] fetch array epr 1599844210152833028-1599846087457767431

---------------------------------------------------------------------
Intel Corporation SAS (French simplified joint stock company)
Registered headquarters: "Les Montalets"- 2, rue de Paris,
92196 Meudon Cedex, France
Registration Number:  302 456 199 R.C.S. NANTERRE
Capital: 4,572,000 Euros

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

Join daos@daos.groups.io to automatically receive all group messages.