Topics

Link Error with DAOS 1.0


Archie Morningstar
 

Hi,

I was trying to build a clean install of DAOS v1.0.0 using the instructions on the DAOS admin guide (https://daos-stack.github.io/admin/installation/) with just the simple commands:
git checkout v1.0.0
git submodule init
git submodule update
scons --config=force --build-deps=yes install

But I'm hitting a linker error. Is there something I'm missing?
I think the libraries are in isal but I'm not familiar with scons to modify the build scripts to add them.

scons: Building targets ...
<lambda>(["build/src/control/bin/daos_admin"], ["src/control/cmd/daos_admin/main.go", "build/src/control/lib/spdk/libnvme_control.a"])
go install -v -ldflags "-X main.daosVersion=1.0.0 -X main.configDir=/home/archie/daos_1.0/daos/install/etc -B 0x10c0ffc53672015e4d3e239af2f447c247b05c02" github.com/daos-stack/daos/src/control/cmd/daos_admin
github.com/daos-stack/daos/src/control/lib/spdk
# github.com/daos-stack/daos/src/control/lib/spdk
/usr/bin/ld: /home/archie/daos_1.0/daos/install/lib/libspdk_env_dpdk.so: undefined reference to `rte_eal_get_configuration'
/usr/bin/ld: /home/archie/daos_1.0/daos/install/lib/libspdk_util.so: undefined reference to `crc16_t10dif'
/usr/bin/ld: /home/archie/daos_1.0/daos/install/lib/libspdk_util.so: undefined reference to `crc16_t10dif_copy'
/usr/bin/ld: /home/archie/daos_1.0/daos/install/lib/libspdk_env_dpdk.so: undefined reference to `lcore_config'
/usr/bin/ld: /home/archie/daos_1.0/daos/install/lib/libspdk_util.so: undefined reference to `crc32_iscsi'
collect2: error: ld returned 1 exit status
scons: *** Error 2
scons: *** [build/src/control/bin/daos_admin] Error 2
scons: building terminated because of errors.

Archie


Olivier, Jeffrey V
 

What is the Linux distro and version?

Composed on a tiny keyboard

On Jun 18, 2020, at 7:04 PM, Archie Morningstar <archiemorningstar88@...> wrote:


Hi,

I was trying to build a clean install of DAOS v1.0.0 using the instructions on the DAOS admin guide (https://daos-stack.github.io/admin/installation/) with just the simple commands:
git checkout v1.0.0
git submodule init
git submodule update
scons --config=force --build-deps=yes install

But I'm hitting a linker error. Is there something I'm missing?
I think the libraries are in isal but I'm not familiar with scons to modify the build scripts to add them.

scons: Building targets ...
<lambda>(["build/src/control/bin/daos_admin"], ["src/control/cmd/daos_admin/main.go", "build/src/control/lib/spdk/libnvme_control.a"])
go install -v -ldflags "-X main.daosVersion=1.0.0 -X main.configDir=/home/archie/daos_1.0/daos/install/etc -B 0x10c0ffc53672015e4d3e239af2f447c247b05c02" github.com/daos-stack/daos/src/control/cmd/daos_admin
github.com/daos-stack/daos/src/control/lib/spdk
# github.com/daos-stack/daos/src/control/lib/spdk
/usr/bin/ld: /home/archie/daos_1.0/daos/install/lib/libspdk_env_dpdk.so: undefined reference to `rte_eal_get_configuration'
/usr/bin/ld: /home/archie/daos_1.0/daos/install/lib/libspdk_util.so: undefined reference to `crc16_t10dif'
/usr/bin/ld: /home/archie/daos_1.0/daos/install/lib/libspdk_util.so: undefined reference to `crc16_t10dif_copy'
/usr/bin/ld: /home/archie/daos_1.0/daos/install/lib/libspdk_env_dpdk.so: undefined reference to `lcore_config'
/usr/bin/ld: /home/archie/daos_1.0/daos/install/lib/libspdk_util.so: undefined reference to `crc32_iscsi'
collect2: error: ld returned 1 exit status
scons: *** Error 2
scons: *** [build/src/control/bin/daos_admin] Error 2
scons: building terminated because of errors.

Archie


Archie Morningstar
 
Edited

Sorry for not having that:
Ubuntu 20.04 LTS

Archie


Olivier, Jeffrey V
 

Hi Archie,

 

I don’t know if 1.0 has been tried with Ubuntu 20.04 though it does work with 18.04.   I will give it a try locally and see what I can find out.

 

-Jeff

 

From: <daos@daos.groups.io> on behalf of Archie Morningstar <archiemorningstar88@...>
Reply-To: "daos@daos.groups.io" <daos@daos.groups.io>
Date: Wednesday, June 24, 2020 at 12:29 PM
To: "daos@daos.groups.io" <daos@daos.groups.io>
Subject: Re: [daos] Link Error with DAOS 1.0

 

[Edited Message Follows]

Sorry for not having that:
Ubuntu 20.04 LTS

Archie


Olivier, Jeffrey V
 

This small patch gets me past that error:

 

jvolivie-desk1:[~/daos_1.0]$ git diff

diff --git a/src/control/lib/spdk/spdk.go b/src/control/lib/spdk/spdk.go

index 28b55e537..bbc3d43c6 100644

--- a/src/control/lib/spdk/spdk.go

+++ b/src/control/lib/spdk/spdk.go

@@ -30,6 +30,7 @@ package spdk

/*

#cgo LDFLAGS: -lspdk_env_dpdk -lrte_mempool -lrte_mempool_ring -lrte_bus_pci

#cgo LDFLAGS: -lrte_pci -lrte_ring -lrte_mbuf -lrte_eal -lrte_kvargs -ldl -lnuma

+#cgo LDFLAGS: -lisal

 #include "stdlib.h"

#include "spdk/stdinc.h"

 

But then it runs into an issue with hwloc.

 

build/src/control/src/github.com/daos-stack/daos/src/control/lib/netdetect/netdetect.go:156:48: could not determine kind of name for C.HWLOC_TOPOLOGY_FLAG_IO_DEVICES

 

It looks like hwloc version 1.5 which should work but perhaps there is something new on Ubuntu 20.04 that we only resolved on master.  

 

-Jeff

 

From: <daos@daos.groups.io> on behalf of "Olivier, Jeffrey V" <jeffrey.v.olivier@...>
Reply-To: "daos@daos.groups.io" <daos@daos.groups.io>
Date: Wednesday, June 24, 2020 at 5:52 PM
To: "daos@daos.groups.io" <daos@daos.groups.io>
Subject: Re: [daos] Link Error with DAOS 1.0

 

Hi Archie,

 

I don’t know if 1.0 has been tried with Ubuntu 20.04 though it does work with 18.04.   I will give it a try locally and see what I can find out.

 

-Jeff

 

From: <daos@daos.groups.io> on behalf of Archie Morningstar <archiemorningstar88@...>
Reply-To: "daos@daos.groups.io" <daos@daos.groups.io>
Date: Wednesday, June 24, 2020 at 12:29 PM
To: "daos@daos.groups.io" <daos@daos.groups.io>
Subject: Re: [daos] Link Error with DAOS 1.0

 

[Edited Message Follows]

Sorry for not having that:
Ubuntu 20.04 LTS

Archie


Nabarro, Tom
 

There have been some fixes in master for build issues with ubuntu 20.04 (specifically for spdk, ipmctl and I think hwloc). Can you try building from master?

 

Tom

 

From: daos@daos.groups.io <daos@daos.groups.io> On Behalf Of Olivier, Jeffrey V
Sent: Thursday, June 25, 2020 2:06 AM
To: daos@daos.groups.io
Subject: Re: [daos] Link Error with DAOS 1.0

 

This small patch gets me past that error:

 

jvolivie-desk1:[~/daos_1.0]$ git diff

diff --git a/src/control/lib/spdk/spdk.go b/src/control/lib/spdk/spdk.go

index 28b55e537..bbc3d43c6 100644

--- a/src/control/lib/spdk/spdk.go

+++ b/src/control/lib/spdk/spdk.go

@@ -30,6 +30,7 @@ package spdk

/*

#cgo LDFLAGS: -lspdk_env_dpdk -lrte_mempool -lrte_mempool_ring -lrte_bus_pci

#cgo LDFLAGS: -lrte_pci -lrte_ring -lrte_mbuf -lrte_eal -lrte_kvargs -ldl -lnuma

+#cgo LDFLAGS: -lisal

 

 #include "stdlib.h"

#include "spdk/stdinc.h"

 

But then it runs into an issue with hwloc.

 

build/src/control/src/github.com/daos-stack/daos/src/control/lib/netdetect/netdetect.go:156:48: could not determine kind of name for C.HWLOC_TOPOLOGY_FLAG_IO_DEVICES

 

It looks like hwloc version 1.5 which should work but perhaps there is something new on Ubuntu 20.04 that we only resolved on master.  

 

-Jeff

 

From: <daos@daos.groups.io> on behalf of "Olivier, Jeffrey V" <jeffrey.v.olivier@...>
Reply-To: "daos@daos.groups.io" <daos@daos.groups.io>
Date: Wednesday, June 24, 2020 at 5:52 PM
To: "daos@daos.groups.io" <daos@daos.groups.io>
Subject: Re: [daos] Link Error with DAOS 1.0

 

Hi Archie,

 

I don’t know if 1.0 has been tried with Ubuntu 20.04 though it does work with 18.04.   I will give it a try locally and see what I can find out.

 

-Jeff

 

From: <daos@daos.groups.io> on behalf of Archie Morningstar <archiemorningstar88@...>
Reply-To: "daos@daos.groups.io" <daos@daos.groups.io>
Date: Wednesday, June 24, 2020 at 12:29 PM
To: "daos@daos.groups.io" <daos@daos.groups.io>
Subject: Re: [daos] Link Error with DAOS 1.0

 

[Edited Message Follows]

Sorry for not having that:
Ubuntu 20.04 LTS

Archie

---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.


Olivier, Jeffrey V
 

H Archie,

 

On the hwloc issue, it looks like Ubuntu 20 has hwloc API version 2.x and the fixes for that version of hwloc are only in master.   Switching to master may be your best bet as Ubuntu isn’t officially supported in 1.0.

 

However, if you really wan to get it building, you can try this.  Before we had support for hwloc 2.x, I added the following workaround on my Fedora box.  I built a 1.5 version of hwloc in /opt/hwloc and added the following to my ~/.scons_localrc

 

Import(‘env’)

env.PrependUnique(LIBPATH=[“/opt/hwloc/lib”]

env.PrependUnique(CPPAPTH=[“/opt/hwloc/include”])

env.PrependUnique(RPATH=[“/opt/hwloc/lib”])

env[“ENV”][“CGO_CFLAGS”] = “-I/opt/hwloc/include”

env[“ENV”][“CGO_LDFLAGS”] = “-L/opt/hwloc/lib -Wl,-rpath=/opt/hwloc/lib”

 

But I would recommend using master instead if you need to use Ubuntu 20.04 as Ubuntu isn’t officially supported in the 1.0 release.

 

-Jeff

 

 

From: <daos@daos.groups.io> on behalf of "Nabarro, Tom" <tom.nabarro@...>
Reply-To: "daos@daos.groups.io" <daos@daos.groups.io>
Date: Thursday, June 25, 2020 at 2:36 AM
To: "daos@daos.groups.io" <daos@daos.groups.io>
Subject: Re: [daos] Link Error with DAOS 1.0

 

There have been some fixes in master for build issues with ubuntu 20.04 (specifically for spdk, ipmctl and I think hwloc). Can you try building from master?

 

Tom

 

From: daos@daos.groups.io <daos@daos.groups.io> On Behalf Of Olivier, Jeffrey V
Sent: Thursday, June 25, 2020 2:06 AM
To: daos@daos.groups.io
Subject: Re: [daos] Link Error with DAOS 1.0

 

This small patch gets me past that error:

 

jvolivie-desk1:[~/daos_1.0]$ git diff

diff --git a/src/control/lib/spdk/spdk.go b/src/control/lib/spdk/spdk.go

index 28b55e537..bbc3d43c6 100644

--- a/src/control/lib/spdk/spdk.go

+++ b/src/control/lib/spdk/spdk.go

@@ -30,6 +30,7 @@ package spdk

/*

#cgo LDFLAGS: -lspdk_env_dpdk -lrte_mempool -lrte_mempool_ring -lrte_bus_pci

#cgo LDFLAGS: -lrte_pci -lrte_ring -lrte_mbuf -lrte_eal -lrte_kvargs -ldl -lnuma

+#cgo LDFLAGS: -lisal

 

 #include "stdlib.h"

#include "spdk/stdinc.h"

 

But then it runs into an issue with hwloc.

 

build/src/control/src/github.com/daos-stack/daos/src/control/lib/netdetect/netdetect.go:156:48: could not determine kind of name for C.HWLOC_TOPOLOGY_FLAG_IO_DEVICES

 

It looks like hwloc version 1.5 which should work but perhaps there is something new on Ubuntu 20.04 that we only resolved on master.  

 

-Jeff

 

From: <daos@daos.groups.io> on behalf of "Olivier, Jeffrey V" <jeffrey.v.olivier@...>
Reply-To: "daos@daos.groups.io" <daos@daos.groups.io>
Date: Wednesday, June 24, 2020 at 5:52 PM
To: "daos@daos.groups.io" <daos@daos.groups.io>
Subject: Re: [daos] Link Error with DAOS 1.0

 

Hi Archie,

 

I don’t know if 1.0 has been tried with Ubuntu 20.04 though it does work with 18.04.   I will give it a try locally and see what I can find out.

 

-Jeff

 

From: <daos@daos.groups.io> on behalf of Archie Morningstar <archiemorningstar88@...>
Reply-To: "daos@daos.groups.io" <daos@daos.groups.io>
Date: Wednesday, June 24, 2020 at 12:29 PM
To: "daos@daos.groups.io" <daos@daos.groups.io>
Subject: Re: [daos] Link Error with DAOS 1.0

 

[Edited Message Follows]

Sorry for not having that:
Ubuntu 20.04 LTS

Archie

---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.