Re: [External] Re: [daos] failed to create pool: -1023 #chat

Shengyu SY19 Zhang

Hello Johann,


Since there is no nvdimm on my system, I’m using tempfs mountted to /mnt/daos as described in the document, and nvme is leaving unformatted, it is using via SPDK.

I can’t post file on the jira, I haven’t got a portal to register on it.

Here is the outpus of the server side :


2019/06/20 18:18:12 config.go:108: debug: DAOS config read from /usr/local/etc/daos_server.yml

2019/06/20 18:18:12 config.go:144: debug: Active config saved to /usr/local/etc/ (read-only)

2019/06/20 18:18:12 config.go:353: debug: Switching control log level to DEBUG

2019/06/20 18:18:12 config.go:368: debug: daos_server logging to file /tmp/daos_control.log

Starting SPDK v18.07-pre / DPDK 18.02.0 initialization...

[ DPDK EAL parameters: spdk -c 0x1 --file-prefix=spdk1929786431 --base-virtaddr=0x200000000000 --proc-type=auto ]

EAL: Detected 20 lcore(s)

EAL: Auto-detected process type: PRIMARY

EAL: No free hugepages reported in hugepages-1048576kB

EAL: Multi-process socket /var/run/.spdk1929786431_unix

EAL: Probing VFIO support...

EAL: VFIO support initialized

EAL: PCI device 0000:03:00.0 on NUMA socket 0

EAL:   probe driver: 8086:953 spdk_nvme

EAL:   using IOMMU type 1 (Type 1)

EAL: PCI device 0000:05:00.0 on NUMA socket 0

EAL:   probe driver: 8086:953 spdk_nvme

no NVDIMMs found!

waiting for storage format on server 0


From: <> On Behalf Of Lombardi, Johann
Sent: Thursday, June 20, 2019 6:01 PM
Subject: Re: [External] Re: [daos] failed to create pool: -1023 #chat




First of all, please note that we are in the process of implementing our own wire-up protocol that will eventually allow us to start the DAOS server on each storage node independently. At that point, we won’t require opmi/pmix any longer and will be able to start the DAOS servers via systemd, kubernetes or any parallel launchers (e.g. pdsh, …). This feature will be available this summer.


Meanwhile, it would be great to see the output of the “orterun … daos_server" command. I suspect that the backend storage hasn’t been formatted and the data plane not started yet. Could you please file a ticket on and attach the daos_server output? Thanks.





From: <> on behalf of Shengyu SY19 Zhang <zhangsy19@...>
Reply-To: "" <>
Date: Thursday 20 June 2019 at 10:35
To: "" <>
Subject: Re: [External] Re: [daos] failed to create pool: -1023 #chat




Thank you for your help, since the ompi/pmix is depends projects of daos, I didnt touch them, just follow quick-start in github, to build and run it.

I enter the path _build.external/ompi and pmix,  and then make && make install, all finished successfully.

For report-uri, since Im running daos server in the /root/daos, I can see the uri file was created and can see the content, therefor just specify full path for the client, seems connection between client and server is OK (if I specify wrong, or nic not started, there is another error).

And also if I run: daosctl create-pool testpool, will get the same issue.

So Im wondering where the problem is.

There are some information in the logs:

PMIx_Lookup group daos_server failed, rc: -46, value.type 0.
crt_pmix_attach group daos_server failed, rc: -1023.



Best Regards

From: <> On Behalf Of xuezhao.liu@...
Sent: Thursday, June 20, 2019 4:08 PM
Subject: [External] Re: [daos] failed to create pool: -1023 #chat


-1023 is DER_PMIX, that commonly due to your ompi/pmix building/usage is incorrect.
You may check "which orterun" to see if the orterun path is the one built by DAOS package.
and for server loading cmd line, at least you need "--report-uri /root/daos/uuu" to allow the dmg client can hook with it (by --ompi-server option).

Intel Corporation SAS (French simplified joint stock company)
Registered headquarters: "Les Montalets"- 2, rue de Paris,
92196 Meudon Cedex, France
Registration Number:  302 456 199 R.C.S. NANTERRE
Capital: 4,572,000 Euros

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

Join { to automatically receive all group messages.