Re: no dRPC client set on Ubuntu 20.04.1


Nabarro, Tom
 

Hello Gert and Lee,

 

https://github.com/daos-stack/daos/pull/3480 has been merged with master to fix this issue.

 

dpdk (used as the default environment for spdk) is using weak symbols when linking:

 

"dpdk/lib/librte_eal/common/include/rte_common.h:#define _rte_weak _attribute__((__weak__)) "

 

and Ubuntu sets --as-needed which means some of the constructor functions in dpdk mempool libs (which handle the access to hugepages) don't get run and cause failure in spdk_thread_lib_init() (called from bio_xstream.c in DAOS).

 

The fix involves setting --no-as-needed linker flags explicitly during build of the relevant bio module and control-plane libs to pull in all symbols regardless of whether they are explicitly called or not.

 

Thanks to Jim Harris in the SPDK for help in resolving the issue. Please confirm the fix works if possible.

 

Regards,

Tom Nabarro – DCG/ESAD

M: +44 (0)7786 260986

Skype: tom.nabarro

 

From: daos@daos.groups.io <daos@daos.groups.io> On Behalf Of Gert Pauwels
Sent: Monday, September 14, 2020 4:30 PM
To: daos@daos.groups.io
Subject: Re: [daos] no dRPC client set on Ubuntu 20.04.1

 

Hi Tom,

After booting I run the following 2 commands:
root@intel-S2600WFD:~/daos# cat /proc/meminfo | grep Huge
AnonHugePages:         0 kB
ShmemHugePages:        0 kB
FileHugePages:         0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
Hugetlb:               0 kB
root@intel-S2600WFD:~/daos# ls -lah /dev/hugepages/
total 0
drwxr-xr-x  2 root root    0 Sep 14 16:40 .
drwxr-xr-x 20 root root 4,3K Sep 14 16:41 ..

 
At this point followed the steps you described to bind the NVMe drives to the kernel and wiped the pmem[01] devices.
After running "dmg storage format" from another tty the error showed and I stopped daos_server.

The daos_server.log, daos_adim.log and daos_control.log are attached.

At this point I run the two commands again:

root@intel-S2600WFD:~/daos# cat /proc/meminfo | grep Huge
AnonHugePages:         0 kB
ShmemHugePages:        0 kB
FileHugePages:         0 kB
HugePages_Total:    4096
HugePages_Free:     4095
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
Hugetlb:         8388608 kB

root@intel-S2600WFD:~/daos# ls -lah /dev/hugepages/spdk_pid4857map_0
-rw------- 1 root root 2,0M Sep 14 16:49 /dev/hugepages/spdk_pid4857map_0


Regards,

Gert,

---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

Join daos@daos.groups.io to automatically receive all group messages.