Unable to build libmfu properly


Bohning, Dalton
 

I’m not sure why the libraries aren’t being linked/loaded – maybe someone more knowledgeable on the DAOS team has an idea. The make command does include the libraries, and the libraries are available, so I wouldn’t think there should be an issue.

 

~Dalton Bohning

 

From: daos@daos.groups.io <daos@daos.groups.io> On Behalf Of netsurfed
Sent: Thursday, December 30, 2021 5:32 PM
To: daos@daos.groups.io
Subject: Re: [daos] Unable to build libmfu properly

 

Yes, the same errors.

daos_agent@sw2:~/git/mpifileutils/build$ export LD_LIBRARY_PATH=${MY_DAOS_INSTALL_PATH}/lib64:$LD_LIBRARY_PATH

daos_agent@sw2:~/git/mpifileutils/build$ echo $LD_LIBRARY_PATH

/home/daos_agent/git/daos/build/lib64:

daos_agent@sw2:~/git/mpifileutils/build$ make

[ 32%] Built target mfu_o

[ 34%] Built target mfu-static

[ 36%] Built target mfu

[ 37%] Linking C executable dbcast

/usr/bin/ld: ../common/libmfu.so: undefined reference to `d_hash_rec_insert'

/usr/bin/ld: ../common/libmfu.so: undefined reference to `dfs_lookup'

/usr/bin/ld: ../common/libmfu.so: undefined reference to `d_hash_rec_unlinked'

/usr/bin/ld: ../common/libmfu.so: undefined reference to `dfs_stat'

/usr/bin/ld: ../common/libmfu.so: undefined reference to `dfs_readdir'

/usr/bin/ld: ../common/libmfu.so: undefined reference to `d_hash_table_create'

/usr/bin/ld: ../common/libmfu.so: undefined reference to `dfs_obj_anchor_split'

/usr/bin/ld: ../common/libmfu.so: undefined reference to `dfs_release'

/usr/bin/ld: ../common/libmfu.so: undefined reference to `d_hash_rec_find'

/usr/bin/ld: ../common/libmfu.so: undefined reference to `dfs_obj_anchor_set'

collect2: error: ld returned 1 exit status

make[2]: *** [src/dbcast/CMakeFiles/dbcast.dir/build.make:90: src/dbcast/dbcast] Error 1

make[1]: *** [CMakeFiles/Makefile2:539: src/dbcast/CMakeFiles/dbcast.dir/all] Error 2

make: *** [Makefile:130: all] Error 2

 


netsurfed
 

Yes, the same errors.

daos_agent@sw2:~/git/mpifileutils/build$ export LD_LIBRARY_PATH=${MY_DAOS_INSTALL_PATH}/lib64:$LD_LIBRARY_PATH
daos_agent@sw2:~/git/mpifileutils/build$ echo $LD_LIBRARY_PATH
/home/daos_agent/git/daos/build/lib64:
daos_agent@sw2:~/git/mpifileutils/build$ make
[ 32%] Built target mfu_o
[ 34%] Built target mfu-static
[ 36%] Built target mfu
[ 37%] Linking C executable dbcast
/usr/bin/ld: ../common/libmfu.so: undefined reference to `d_hash_rec_insert'
/usr/bin/ld: ../common/libmfu.so: undefined reference to `dfs_lookup'
/usr/bin/ld: ../common/libmfu.so: undefined reference to `d_hash_rec_unlinked'
/usr/bin/ld: ../common/libmfu.so: undefined reference to `dfs_stat'
/usr/bin/ld: ../common/libmfu.so: undefined reference to `dfs_readdir'
/usr/bin/ld: ../common/libmfu.so: undefined reference to `d_hash_table_create'
/usr/bin/ld: ../common/libmfu.so: undefined reference to `dfs_obj_anchor_split'
/usr/bin/ld: ../common/libmfu.so: undefined reference to `dfs_release'
/usr/bin/ld: ../common/libmfu.so: undefined reference to `d_hash_rec_find'
/usr/bin/ld: ../common/libmfu.so: undefined reference to `dfs_obj_anchor_set'
collect2: error: ld returned 1 exit status
make[2]: *** [src/dbcast/CMakeFiles/dbcast.dir/build.make:90: src/dbcast/dbcast] Error 1
make[1]: *** [CMakeFiles/Makefile2:539: src/dbcast/CMakeFiles/dbcast.dir/all] Error 2
make: *** [Makefile:130: all] Error 2
 


Bohning, Dalton
 

Will you try appending ${MY_DAOS_INSTALL_PATH}/lib64 to your LD_LIBRARY_PATH and re-run the “make” command?

export LD_LIBRARY_PATH=${MY_DAOS_INSTALL_PATH}/lib64:$LD_LIBRARY_PATH

 

~Dalton Bohning

 

From: daos@daos.groups.io <daos@daos.groups.io> On Behalf Of netsurfed
Sent: Wednesday, December 29, 2021 6:04 PM
To: daos@daos.groups.io
Subject: Re: [daos] Unable to build libmfu properly

 

Hi, 

$ cat ${MY_MFU_BUILD_PATH}/src/common/CMakeFiles/mfu.dir/link.txt

/usr/bin/cc -fPIC -I/home/daos_agent/git/daos/build/include -L/home/daos_agent/git/daos/build/lib64/ -luuid -ldaos -ldfs -ldaos_common -lgurt -lpthread -shared -Wl,-soname,libmfu.so -o libmfu.so CMakeFiles/mfu_o.dir/mfu_bz2.c.o CMakeFiles/mfu_o.dir/mfu_bz2_static.c.o CMakeFiles/mfu_o.dir/mfu_compress_bz2_libcircle.c.o CMakeFiles/mfu_o.dir/mfu_decompress_bz2_libcircle.c.o CMakeFiles/mfu_o.dir/mfu_flist.c.o CMakeFiles/mfu_o.dir/mfu_flist_chunk.c.o CMakeFiles/mfu_o.dir/mfu_flist_copy.c.o CMakeFiles/mfu_o.dir/mfu_flist_io.c.o CMakeFiles/mfu_o.dir/mfu_flist_chmod.c.o CMakeFiles/mfu_o.dir/mfu_flist_create.c.o CMakeFiles/mfu_o.dir/mfu_flist_remove.c.o CMakeFiles/mfu_o.dir/mfu_flist_sort.c.o CMakeFiles/mfu_o.dir/mfu_flist_usrgrp.c.o CMakeFiles/mfu_o.dir/mfu_flist_walk.c.o CMakeFiles/mfu_o.dir/mfu_io.c.o CMakeFiles/mfu_o.dir/mfu_param_path.c.o CMakeFiles/mfu_o.dir/mfu_path.c.o CMakeFiles/mfu_o.dir/mfu_pred.c.o CMakeFiles/mfu_o.dir/mfu_util.c.o CMakeFiles/mfu_o.dir/strmap.c.o  -Wl,-rpath,/usr/lib/x86_64-linux-gnu/openmpi/lib:/home/daos_agent/install/lib: /usr/lib/x86_64-linux-gnu/openmpi/lib/libmpi.so /home/daos_agent/install/lib/libdtcmp.so -larchive /home/daos_agent/install/lib/libcircle.so -lbz2

 

And The build command of libmfu is:
git clone 
https://github.com/mchaarawi/mpifileutils -b pfind_integration "${MY_MFU_SOURCE_PATH}" &&
mkdir -p "${MY_MFU_BUILD_PATH}" &&
cd "${MY_MFU_BUILD_PATH}" &&
CFLAGS="-I${MY_DAOS_INSTALL_PATH}/include" \
LDFLAGS="-L${MY_DAOS_INSTALL_PATH}/lib64/ -luuid -ldaos -ldfs -ldaos_common -lgurt -lpthread" \
cmake "${MY_MFU_SOURCE_PATH}" \
-DENABLE_XATTRS=OFF \
-DWITH_DTCMP_PREFIX=${MY_MFU_INSTALL_PATH} \
-DWITH_LibCircle_PREFIX=${MY_MFU_INSTALL_PATH} \
-DCMAKE_INSTALL_PREFIX=${MY_MFU_INSTALL_PATH} &&
make -j8 install


netsurfed
 

Hi, 
$ cat ${MY_MFU_BUILD_PATH}/src/common/CMakeFiles/mfu.dir/link.txt
/usr/bin/cc -fPIC -I/home/daos_agent/git/daos/build/include -L/home/daos_agent/git/daos/build/lib64/ -luuid -ldaos -ldfs -ldaos_common -lgurt -lpthread -shared -Wl,-soname,libmfu.so -o libmfu.so CMakeFiles/mfu_o.dir/mfu_bz2.c.o CMakeFiles/mfu_o.dir/mfu_bz2_static.c.o CMakeFiles/mfu_o.dir/mfu_compress_bz2_libcircle.c.o CMakeFiles/mfu_o.dir/mfu_decompress_bz2_libcircle.c.o CMakeFiles/mfu_o.dir/mfu_flist.c.o CMakeFiles/mfu_o.dir/mfu_flist_chunk.c.o CMakeFiles/mfu_o.dir/mfu_flist_copy.c.o CMakeFiles/mfu_o.dir/mfu_flist_io.c.o CMakeFiles/mfu_o.dir/mfu_flist_chmod.c.o CMakeFiles/mfu_o.dir/mfu_flist_create.c.o CMakeFiles/mfu_o.dir/mfu_flist_remove.c.o CMakeFiles/mfu_o.dir/mfu_flist_sort.c.o CMakeFiles/mfu_o.dir/mfu_flist_usrgrp.c.o CMakeFiles/mfu_o.dir/mfu_flist_walk.c.o CMakeFiles/mfu_o.dir/mfu_io.c.o CMakeFiles/mfu_o.dir/mfu_param_path.c.o CMakeFiles/mfu_o.dir/mfu_path.c.o CMakeFiles/mfu_o.dir/mfu_pred.c.o CMakeFiles/mfu_o.dir/mfu_util.c.o CMakeFiles/mfu_o.dir/strmap.c.o  -Wl,-rpath,/usr/lib/x86_64-linux-gnu/openmpi/lib:/home/daos_agent/install/lib: /usr/lib/x86_64-linux-gnu/openmpi/lib/libmpi.so /home/daos_agent/install/lib/libdtcmp.so -larchive /home/daos_agent/install/lib/libcircle.so -lbz2
 
And The build command of libmfu is:
git clone https://github.com/mchaarawi/mpifileutils -b pfind_integration "${MY_MFU_SOURCE_PATH}" &&
mkdir -p "${MY_MFU_BUILD_PATH}" &&
cd "${MY_MFU_BUILD_PATH}" &&
CFLAGS="-I${MY_DAOS_INSTALL_PATH}/include" \
LDFLAGS="-L${MY_DAOS_INSTALL_PATH}/lib64/ -luuid -ldaos -ldfs -ldaos_common -lgurt -lpthread" \
cmake "${MY_MFU_SOURCE_PATH}" \
-DENABLE_XATTRS=OFF \
-DWITH_DTCMP_PREFIX=${MY_MFU_INSTALL_PATH} \
-DWITH_LibCircle_PREFIX=${MY_MFU_INSTALL_PATH} \
-DCMAKE_INSTALL_PREFIX=${MY_MFU_INSTALL_PATH} &&
make -j8 install


Bohning, Dalton
 

Interesting. As you said, the libraries are in fact there. And I think cmake is finding the libraries, or else cmake would print something like “/usr/bin/ld: cannot find -ldfs” before getting to the “make” command. Could you please the compile command used to build libmfu?

$ cat ${MY_MFU_BUILD_PATH}/src/common/CMakeFiles/mfu.dir/link.txt

 

~Dalton Bohning

From: daos@daos.groups.io <daos@daos.groups.io> On Behalf Of netsurfed
Sent: Tuesday, December 28, 2021 9:40 PM
To: daos@daos.groups.io
Subject: Re: [daos] Unable to build libmfu properly

 

Hi, yes, "build" is where daos is installed.

$ ls -l /home/daos_agent/git/daos/build/lib64/

total 22080

drwxrwxr-x 3 daos_agent daos_agent    4096 Dec 13 08:06 daos

drwxrwxr-x 2 daos_agent daos_agent    4096 Dec 13 10:18 daos_srv

lrwxrwxrwx 1 daos_agent daos_agent      16 Dec 13 10:18 libcart.so -> libcart.so.4.9.0

lrwxrwxrwx 1 daos_agent daos_agent      16 Dec 13 10:18 libcart.so.4 -> libcart.so.4.9.0

-rwxrwxr-x 1 daos_agent daos_agent 4948104 Dec 13 10:10 libcart.so.4.9.0

-rwxrwxr-x 1 daos_agent daos_agent  710680 Dec 13 10:10 libdaos_cmd_hdlrs.so

-rwxrwxr-x 1 daos_agent daos_agent 2955992 Dec 13 10:10 libdaos_common_pmem.so

-rwxrwxr-x 1 daos_agent daos_agent 2905240 Dec 13 10:10 libdaos_common.so

lrwxrwxrwx 1 daos_agent daos_agent      16 Dec 13 10:18 libdaos.so -> libdaos.so.2.0.0

lrwxrwxrwx 1 daos_agent daos_agent      16 Dec 13 10:18 libdaos.so.2 -> libdaos.so.2.0.0

-rwxrwxr-x 1 daos_agent daos_agent 6814552 Dec 13 10:04 libdaos.so.2.0.0

-rwxrwxr-x 1 daos_agent daos_agent  274776 Dec 13 10:10 libdaos_tests.so

-rwxrwxr-x 1 daos_agent daos_agent   27160 Dec 13 10:18 libdfs_internal.so

-rwxrwxr-x 1 daos_agent daos_agent  816728 Dec 13 10:10 libdfs.so

-rw-rw-r-- 1 daos_agent daos_agent  206304 Dec 13 08:07 libdfuse.a

-rwxrwxr-x 1 daos_agent daos_agent  137440 Dec 13 10:18 libdfuse.so

-rwxrwxr-x 1 daos_agent daos_agent  116120 Dec 13 10:10 libdts.so

-rwxrwxr-x 1 daos_agent daos_agent  158712 Dec 13 10:10 libduns.so

lrwxrwxrwx 1 daos_agent daos_agent      16 Dec 13 10:18 libgurt.so -> libgurt.so.4.9.0

lrwxrwxrwx 1 daos_agent daos_agent      16 Dec 13 10:18 libgurt.so.4 -> libgurt.so.4.9.0

-rwxrwxr-x 1 daos_agent daos_agent 1132488 Dec 13 10:03 libgurt.so.4.9.0

-rw-rw-r-- 1 daos_agent daos_agent  777432 Dec 13 08:06 libioil.a

-rwxrwxr-x 1 daos_agent daos_agent  411776 Dec 13 10:18 libioil.so

-rw-rw-r-- 1 daos_agent daos_agent  169190 Dec 13 08:03 libnvme_control.a

drwxrwxr-x 3 daos_agent daos_agent    4096 Dec 13 08:07 python3.8


And "${MY_DAOS_INSTALL_PATH}/include" is the installed include directory.

$ ls -l ${MY_DAOS_INSTALL_PATH}/include

total 332

drwxrwxr-x 2 daos_agent daos_agent  4096 Dec 17 08:11 cart

drwxrwxr-x 2 daos_agent daos_agent  4096 Dec 13 08:06 daos

-rw-r--r-- 1 daos_agent daos_agent  6274 Dec 13 06:39 daos_api.h

-rw-r--r-- 1 daos_agent daos_agent 14770 Dec 13 06:39 daos_array.h

-rw-r--r-- 1 daos_agent daos_agent 27543 Dec 13 06:39 daos_cont.h

-rw-r--r-- 1 daos_agent daos_agent 13947 Dec 13 06:39 daos_errno.h

-rw-r--r-- 1 daos_agent daos_agent  8573 Dec 13 06:39 daos_event.h

-rw-r--r-- 1 daos_agent daos_agent 32290 Dec 13 06:39 daos_fs.h

-rw-r--r-- 1 daos_agent daos_agent 16091 Dec 13 06:39 daos_fs_sys.h

-rw-r--r-- 1 daos_agent daos_agent   974 Dec 13 06:39 daos.h

-rw-r--r-- 1 daos_agent daos_agent  6895 Dec 13 06:39 daos_kv.h

-rw-r--r-- 1 daos_agent daos_agent  2990 Dec 13 06:39 daos_mgmt.h

-rw-r--r-- 1 daos_agent daos_agent 17727 Dec 13 06:39 daos_obj_class.h

-rw-r--r-- 1 daos_agent daos_agent 39305 Dec 13 06:39 daos_obj.h

-rw-r--r-- 1 daos_agent daos_agent 14316 Dec 13 06:39 daos_pool.h

-rw-r--r-- 1 daos_agent daos_agent 16599 Dec 13 06:39 daos_prop.h

-rw-r--r-- 1 daos_agent daos_agent 17364 Dec 13 06:39 daos_security.h

drwxrwxr-x 2 daos_agent daos_agent  4096 Dec 13 08:06 daos_srv

-rw-r--r-- 1 daos_agent daos_agent 27072 Dec 13 06:39 daos_task.h

-rw-r--r-- 1 daos_agent daos_agent  4872 Dec 13 06:39 daos_types.h

-rw-r--r-- 1 daos_agent daos_agent  8607 Dec 13 06:39 daos_uns.h

-rw-rw-r-- 1 daos_agent daos_agent   486 Dec 13 08:02 daos_version.h

drwxrwxr-x 2 daos_agent daos_agent  4096 Dec 20 02:12 gurt

drwxrwxr-x 2 daos_agent daos_agent  4096 Dec 13 08:06 spdk


My command to compile daos is "scons PREFIX=./build install --build-deps=yes --config=force".


netsurfed
 

Hi, yes, "build" is where daos is installed.
$ ls -l /home/daos_agent/git/daos/build/lib64/
total 22080
drwxrwxr-x 3 daos_agent daos_agent    4096 Dec 13 08:06 daos
drwxrwxr-x 2 daos_agent daos_agent    4096 Dec 13 10:18 daos_srv
lrwxrwxrwx 1 daos_agent daos_agent      16 Dec 13 10:18 libcart.so -> libcart.so.4.9.0
lrwxrwxrwx 1 daos_agent daos_agent      16 Dec 13 10:18 libcart.so.4 -> libcart.so.4.9.0
-rwxrwxr-x 1 daos_agent daos_agent 4948104 Dec 13 10:10 libcart.so.4.9.0
-rwxrwxr-x 1 daos_agent daos_agent  710680 Dec 13 10:10 libdaos_cmd_hdlrs.so
-rwxrwxr-x 1 daos_agent daos_agent 2955992 Dec 13 10:10 libdaos_common_pmem.so
-rwxrwxr-x 1 daos_agent daos_agent 2905240 Dec 13 10:10 libdaos_common.so
lrwxrwxrwx 1 daos_agent daos_agent      16 Dec 13 10:18 libdaos.so -> libdaos.so.2.0.0
lrwxrwxrwx 1 daos_agent daos_agent      16 Dec 13 10:18 libdaos.so.2 -> libdaos.so.2.0.0
-rwxrwxr-x 1 daos_agent daos_agent 6814552 Dec 13 10:04 libdaos.so.2.0.0
-rwxrwxr-x 1 daos_agent daos_agent  274776 Dec 13 10:10 libdaos_tests.so
-rwxrwxr-x 1 daos_agent daos_agent   27160 Dec 13 10:18 libdfs_internal.so
-rwxrwxr-x 1 daos_agent daos_agent  816728 Dec 13 10:10 libdfs.so
-rw-rw-r-- 1 daos_agent daos_agent  206304 Dec 13 08:07 libdfuse.a
-rwxrwxr-x 1 daos_agent daos_agent  137440 Dec 13 10:18 libdfuse.so
-rwxrwxr-x 1 daos_agent daos_agent  116120 Dec 13 10:10 libdts.so
-rwxrwxr-x 1 daos_agent daos_agent  158712 Dec 13 10:10 libduns.so
lrwxrwxrwx 1 daos_agent daos_agent      16 Dec 13 10:18 libgurt.so -> libgurt.so.4.9.0
lrwxrwxrwx 1 daos_agent daos_agent      16 Dec 13 10:18 libgurt.so.4 -> libgurt.so.4.9.0
-rwxrwxr-x 1 daos_agent daos_agent 1132488 Dec 13 10:03 libgurt.so.4.9.0
-rw-rw-r-- 1 daos_agent daos_agent  777432 Dec 13 08:06 libioil.a
-rwxrwxr-x 1 daos_agent daos_agent  411776 Dec 13 10:18 libioil.so
-rw-rw-r-- 1 daos_agent daos_agent  169190 Dec 13 08:03 libnvme_control.a
drwxrwxr-x 3 daos_agent daos_agent    4096 Dec 13 08:07 python3.8

And "${MY_DAOS_INSTALL_PATH}/include" is the installed include directory.
$ ls -l ${MY_DAOS_INSTALL_PATH}/include
total 332
drwxrwxr-x 2 daos_agent daos_agent  4096 Dec 17 08:11 cart
drwxrwxr-x 2 daos_agent daos_agent  4096 Dec 13 08:06 daos
-rw-r--r-- 1 daos_agent daos_agent  6274 Dec 13 06:39 daos_api.h
-rw-r--r-- 1 daos_agent daos_agent 14770 Dec 13 06:39 daos_array.h
-rw-r--r-- 1 daos_agent daos_agent 27543 Dec 13 06:39 daos_cont.h
-rw-r--r-- 1 daos_agent daos_agent 13947 Dec 13 06:39 daos_errno.h
-rw-r--r-- 1 daos_agent daos_agent  8573 Dec 13 06:39 daos_event.h
-rw-r--r-- 1 daos_agent daos_agent 32290 Dec 13 06:39 daos_fs.h
-rw-r--r-- 1 daos_agent daos_agent 16091 Dec 13 06:39 daos_fs_sys.h
-rw-r--r-- 1 daos_agent daos_agent   974 Dec 13 06:39 daos.h
-rw-r--r-- 1 daos_agent daos_agent  6895 Dec 13 06:39 daos_kv.h
-rw-r--r-- 1 daos_agent daos_agent  2990 Dec 13 06:39 daos_mgmt.h
-rw-r--r-- 1 daos_agent daos_agent 17727 Dec 13 06:39 daos_obj_class.h
-rw-r--r-- 1 daos_agent daos_agent 39305 Dec 13 06:39 daos_obj.h
-rw-r--r-- 1 daos_agent daos_agent 14316 Dec 13 06:39 daos_pool.h
-rw-r--r-- 1 daos_agent daos_agent 16599 Dec 13 06:39 daos_prop.h
-rw-r--r-- 1 daos_agent daos_agent 17364 Dec 13 06:39 daos_security.h
drwxrwxr-x 2 daos_agent daos_agent  4096 Dec 13 08:06 daos_srv
-rw-r--r-- 1 daos_agent daos_agent 27072 Dec 13 06:39 daos_task.h
-rw-r--r-- 1 daos_agent daos_agent  4872 Dec 13 06:39 daos_types.h
-rw-r--r-- 1 daos_agent daos_agent  8607 Dec 13 06:39 daos_uns.h
-rw-rw-r-- 1 daos_agent daos_agent   486 Dec 13 08:02 daos_version.h
drwxrwxr-x 2 daos_agent daos_agent  4096 Dec 20 02:12 gurt
drwxrwxr-x 2 daos_agent daos_agent  4096 Dec 13 08:06 spdk

My command to compile daos is "scons PREFIX=./build install --build-deps=yes --config=force".


Bohning, Dalton
 

Is this directory correct?

/home/daos_agent/git/daos/build/lib64/

Libraries are usually in the “install” directory, not “build”. Is “build” in fact where daos is installed?

 

Similarly, is ${MY_DAOS_INSTALL_PATH}/include the installed include directory?

 

~Dalton Bohning

 

From: daos@daos.groups.io <daos@daos.groups.io> On Behalf Of netsurfed
Sent: Monday, December 27, 2021 9:38 PM
To: daos@daos.groups.io
Subject: Re: [daos] Unable to build libmfu properly

 

On Tue, Dec 28, 2021 at 03:13 AM, Bohning, Dalton wrote:

grep -B 2 "CMAKE_SHARED_LINKER_FLAGS:" ${MY_MFU_BUILD_PATH}/CMakeCache.txt

Hi, I run your command and it prints as follows:
$ grep -B 2 "CMAKE_SHARED_LINKER_FLAGS:" ${MY_MFU_BUILD_PATH}/CMakeCache.txt

//Flags used by the linker during the creation of shared libraries

// during all build types.

CMAKE_SHARED_LINKER_FLAGS:STRING=-L/home/daos_agent/git/daos/build/lib64/ -luuid -ldaos -ldfs -ldaos_common -lgurt -lpthread

 


netsurfed
 

On Tue, Dec 28, 2021 at 03:13 AM, Bohning, Dalton wrote:
grep -B 2 "CMAKE_SHARED_LINKER_FLAGS:" ${MY_MFU_BUILD_PATH}/CMakeCache.txt
Hi, I run your command and it prints as follows:
$ grep -B 2 "CMAKE_SHARED_LINKER_FLAGS:" ${MY_MFU_BUILD_PATH}/CMakeCache.txt
//Flags used by the linker during the creation of shared libraries
// during all build types.
CMAKE_SHARED_LINKER_FLAGS:STRING=-L/home/daos_agent/git/daos/build/lib64/ -luuid -ldaos -ldfs -ldaos_common -lgurt -lpthread
 


Bohning, Dalton
 

Apologies, my previous command snippet was muddled. Correction:

$ grep -B 2 "CMAKE_SHARED_LINKER_FLAGS:" ${MY_MFU_BUILD_PATH}/CMakeCache.txt

//Flags used by the linker during the creation of shared libraries

// during all build types.

CMAKE_SHARED_LINKER_FLAGS:STRING=-L/home/dbohning/daos/install/lib64/ -luuid -ldaos -ldfs -ldaos_common -lgurt -lpthread

 

~Dalton Bohning

 

 

From: Bohning, Dalton
Sent: Monday, December 27, 2021 8:50 AM
To: daos@daos.groups.io
Subject: RE: [daos] Unable to build libmfu properly

 

Hello,

 

It looks like most  of the libraries specified by “-luuid -ldaos -ldfs -ldaos_common -lgurt -lpthread” are not linked. Something to keep in mind is that “CFLAGS=… LDFLAGS=… cmake…” should be executed as a single command so CFLAGS and LDFLAGS is propagated to the cmake environment. You can check this by running:

$ grep -B 1 "CMAKE_EXE_LINKER_FLAGS:" ${MY_MFU_BUILD_PATH}/CMakeCache.txt

grep -B 2 "CMAKE_SHARED_LINKER_FLAGS:" CMakeCache.txt

//Flags used by the linker during the creation of shared libraries

// during all build types.

CMAKE_SHARED_LINKER_FLAGS:STRING=-L/home/dbohning/daos/install/lib64/ -luuid -ldaos -ldfs -ldaos_common -lgurt -lpthread

 

Where CMAKE_SHARED_LINKER_FLAGS should be set to the LDFLAGS from the build command.

 

~Dalton Bohning

 

 

From: daos@daos.groups.io <daos@daos.groups.io> On Behalf Of netsurfed
Sent: Thursday, December 23, 2021 7:11 PM
To: daos@daos.groups.io
Subject: [daos] Unable to build libmfu properly

 

Hi,
I built MFU according to https://daosio.atlassian.net/wiki/spaces/DC/pages/4874571083/IO-500+ISC21 but encountered some "undefined reference" errors.
[ 39%] Linking C executable dbcast
/usr/bin/ld: ../common/libmfu.so: undefined reference to `d_hash_rec_insert'
/usr/bin/ld: ../common/libmfu.so: undefined reference to `dfs_lookup'
/usr/bin/ld: ../common/libmfu.so: undefined reference to `d_hash_rec_unlinked'
/usr/bin/ld: ../common/libmfu.so: undefined reference to `dfs_stat'
/usr/bin/ld: ../common/libmfu.so: undefined reference to `dfs_readdir'
/usr/bin/ld: ../common/libmfu.so: undefined reference to `d_hash_table_create'
/usr/bin/ld: ../common/libmfu.so: undefined reference to `dfs_obj_anchor_split'
/usr/bin/ld: ../common/libmfu.so: undefined reference to `dfs_release'
/usr/bin/ld: ../common/libmfu.so: undefined reference to `d_hash_rec_find'
/usr/bin/ld: ../common/libmfu.so: undefined reference to `dfs_obj_anchor_set'

The build command is:
git clone https://github.com/mchaarawi/mpifileutils -b pfind_integration "${MY_MFU_SOURCE_PATH}" &&
mkdir -p "${MY_MFU_BUILD_PATH}" &&
cd "${MY_MFU_BUILD_PATH}" &&
CFLAGS="-I${MY_DAOS_INSTALL_PATH}/include" \
LDFLAGS="-L${MY_DAOS_INSTALL_PATH}/lib64/ -luuid -ldaos -ldfs -ldaos_common -lgurt -lpthread" \
cmake "${MY_MFU_SOURCE_PATH}" \
-DENABLE_XATTRS=OFF \
-DWITH_DTCMP_PREFIX=${MY_MFU_INSTALL_PATH} \
-DWITH_LibCircle_PREFIX=${MY_MFU_INSTALL_PATH} \
-DCMAKE_INSTALL_PREFIX=${MY_MFU_INSTALL_PATH} &&
make -j8 install

And there is libgurt.so in ${MY_DAOS_INSTALL_PATH}/lib64/.
$ nm -D ${MY_DAOS_INSTALL_PATH}/lib64/libgurt.so | grep d_hash_rec_insert
000000000000fb40 T d_hash_rec_insert
0000000000010100 T d_hash_rec_insert_anonym

But looks like libgurt.so is not linked by libmfu.so.
$ ldd src/common/libmfu.so
linux-vdso.so.1 (0x00007ffda13f3000)
libmpi.so.40 => /lib/x86_64-linux-gnu/libmpi.so.40 (0x00007fd75c830000)
libdtcmp.so => /home/daos_agent/install/lib/libdtcmp.so (0x00007fd75c814000)
libcircle.so.2 => /home/daos_agent/install/lib/libcircle.so.2 (0x00007fd75c806000)
libbz2.so.1.0 => /lib/x86_64-linux-gnu/libbz2.so.1.0 (0x00007fd75c7f3000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fd75c601000)
libopen-rte.so.40 => /lib/x86_64-linux-gnu/libopen-rte.so.40 (0x00007fd75c545000)
libopen-pal.so.40 => /lib/x86_64-linux-gnu/libopen-pal.so.40 (0x00007fd75c497000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fd75c348000)
libhwloc.so.15 => /lib/x86_64-linux-gnu/libhwloc.so.15 (0x00007fd75c2f7000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fd75c2d4000)
liblwgrp.so => /home/daos_agent/install/lib/liblwgrp.so (0x00007fd75c2c5000)
/lib64/ld-linux-x86-64.so.2 (0x00007fd75c9b2000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fd75c2a7000)
libevent-2.1.so.7 => /lib/x86_64-linux-gnu/libevent-2.1.so.7 (0x00007fd75c251000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fd75c24b000)
libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007fd75c246000)
libevent_pthreads-2.1.so.7 => /lib/x86_64-linux-gnu/libevent_pthreads-2.1.so.7 (0x00007fd75c241000)
libudev.so.1 => /lib/x86_64-linux-gnu/libudev.so.1 (0x00007fd75c214000)
libltdl.so.7 => /lib/x86_64-linux-gnu/libltdl.so.7 (0x00007fd75c207000)"


Bohning, Dalton
 

Hello,

 

It looks like most  of the libraries specified by “-luuid -ldaos -ldfs -ldaos_common -lgurt -lpthread” are not linked. Something to keep in mind is that “CFLAGS=… LDFLAGS=… cmake…” should be executed as a single command so CFLAGS and LDFLAGS is propagated to the cmake environment. You can check this by running:

$ grep -B 1 "CMAKE_EXE_LINKER_FLAGS:" ${MY_MFU_BUILD_PATH}/CMakeCache.txt

grep -B 2 "CMAKE_SHARED_LINKER_FLAGS:" CMakeCache.txt

//Flags used by the linker during the creation of shared libraries

// during all build types.

CMAKE_SHARED_LINKER_FLAGS:STRING=-L/home/dbohning/daos/install/lib64/ -luuid -ldaos -ldfs -ldaos_common -lgurt -lpthread

 

Where CMAKE_SHARED_LINKER_FLAGS should be set to the LDFLAGS from the build command.

 

~Dalton Bohning

 

 

From: daos@daos.groups.io <daos@daos.groups.io> On Behalf Of netsurfed
Sent: Thursday, December 23, 2021 7:11 PM
To: daos@daos.groups.io
Subject: [daos] Unable to build libmfu properly

 

Hi,
I built MFU according to https://daosio.atlassian.net/wiki/spaces/DC/pages/4874571083/IO-500+ISC21 but encountered some "undefined reference" errors.
[ 39%] Linking C executable dbcast
/usr/bin/ld: ../common/libmfu.so: undefined reference to `d_hash_rec_insert'
/usr/bin/ld: ../common/libmfu.so: undefined reference to `dfs_lookup'
/usr/bin/ld: ../common/libmfu.so: undefined reference to `d_hash_rec_unlinked'
/usr/bin/ld: ../common/libmfu.so: undefined reference to `dfs_stat'
/usr/bin/ld: ../common/libmfu.so: undefined reference to `dfs_readdir'
/usr/bin/ld: ../common/libmfu.so: undefined reference to `d_hash_table_create'
/usr/bin/ld: ../common/libmfu.so: undefined reference to `dfs_obj_anchor_split'
/usr/bin/ld: ../common/libmfu.so: undefined reference to `dfs_release'
/usr/bin/ld: ../common/libmfu.so: undefined reference to `d_hash_rec_find'
/usr/bin/ld: ../common/libmfu.so: undefined reference to `dfs_obj_anchor_set'

The build command is:
git clone https://github.com/mchaarawi/mpifileutils -b pfind_integration "${MY_MFU_SOURCE_PATH}" &&
mkdir -p "${MY_MFU_BUILD_PATH}" &&
cd "${MY_MFU_BUILD_PATH}" &&
CFLAGS="-I${MY_DAOS_INSTALL_PATH}/include" \
LDFLAGS="-L${MY_DAOS_INSTALL_PATH}/lib64/ -luuid -ldaos -ldfs -ldaos_common -lgurt -lpthread" \
cmake "${MY_MFU_SOURCE_PATH}" \
-DENABLE_XATTRS=OFF \
-DWITH_DTCMP_PREFIX=${MY_MFU_INSTALL_PATH} \
-DWITH_LibCircle_PREFIX=${MY_MFU_INSTALL_PATH} \
-DCMAKE_INSTALL_PREFIX=${MY_MFU_INSTALL_PATH} &&
make -j8 install

And there is libgurt.so in ${MY_DAOS_INSTALL_PATH}/lib64/.
$ nm -D ${MY_DAOS_INSTALL_PATH}/lib64/libgurt.so | grep d_hash_rec_insert
000000000000fb40 T d_hash_rec_insert
0000000000010100 T d_hash_rec_insert_anonym

But looks like libgurt.so is not linked by libmfu.so.
$ ldd src/common/libmfu.so
linux-vdso.so.1 (0x00007ffda13f3000)
libmpi.so.40 => /lib/x86_64-linux-gnu/libmpi.so.40 (0x00007fd75c830000)
libdtcmp.so => /home/daos_agent/install/lib/libdtcmp.so (0x00007fd75c814000)
libcircle.so.2 => /home/daos_agent/install/lib/libcircle.so.2 (0x00007fd75c806000)
libbz2.so.1.0 => /lib/x86_64-linux-gnu/libbz2.so.1.0 (0x00007fd75c7f3000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fd75c601000)
libopen-rte.so.40 => /lib/x86_64-linux-gnu/libopen-rte.so.40 (0x00007fd75c545000)
libopen-pal.so.40 => /lib/x86_64-linux-gnu/libopen-pal.so.40 (0x00007fd75c497000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fd75c348000)
libhwloc.so.15 => /lib/x86_64-linux-gnu/libhwloc.so.15 (0x00007fd75c2f7000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fd75c2d4000)
liblwgrp.so => /home/daos_agent/install/lib/liblwgrp.so (0x00007fd75c2c5000)
/lib64/ld-linux-x86-64.so.2 (0x00007fd75c9b2000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fd75c2a7000)
libevent-2.1.so.7 => /lib/x86_64-linux-gnu/libevent-2.1.so.7 (0x00007fd75c251000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fd75c24b000)
libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007fd75c246000)
libevent_pthreads-2.1.so.7 => /lib/x86_64-linux-gnu/libevent_pthreads-2.1.so.7 (0x00007fd75c241000)
libudev.so.1 => /lib/x86_64-linux-gnu/libudev.so.1 (0x00007fd75c214000)
libltdl.so.7 => /lib/x86_64-linux-gnu/libltdl.so.7 (0x00007fd75c207000)"


netsurfed
 

Hi,
I built MFU according to https://daosio.atlassian.net/wiki/spaces/DC/pages/4874571083/IO-500+ISC21 but encountered some "undefined reference" errors.
[ 39%] Linking C executable dbcast
/usr/bin/ld: ../common/libmfu.so: undefined reference to `d_hash_rec_insert'
/usr/bin/ld: ../common/libmfu.so: undefined reference to `dfs_lookup'
/usr/bin/ld: ../common/libmfu.so: undefined reference to `d_hash_rec_unlinked'
/usr/bin/ld: ../common/libmfu.so: undefined reference to `dfs_stat'
/usr/bin/ld: ../common/libmfu.so: undefined reference to `dfs_readdir'
/usr/bin/ld: ../common/libmfu.so: undefined reference to `d_hash_table_create'
/usr/bin/ld: ../common/libmfu.so: undefined reference to `dfs_obj_anchor_split'
/usr/bin/ld: ../common/libmfu.so: undefined reference to `dfs_release'
/usr/bin/ld: ../common/libmfu.so: undefined reference to `d_hash_rec_find'
/usr/bin/ld: ../common/libmfu.so: undefined reference to `dfs_obj_anchor_set'

The build command is:
git clone https://github.com/mchaarawi/mpifileutils -b pfind_integration "${MY_MFU_SOURCE_PATH}" &&
mkdir -p "${MY_MFU_BUILD_PATH}" &&
cd "${MY_MFU_BUILD_PATH}" &&
CFLAGS="-I${MY_DAOS_INSTALL_PATH}/include" \
LDFLAGS="-L${MY_DAOS_INSTALL_PATH}/lib64/ -luuid -ldaos -ldfs -ldaos_common -lgurt -lpthread" \
cmake "${MY_MFU_SOURCE_PATH}" \
-DENABLE_XATTRS=OFF \
-DWITH_DTCMP_PREFIX=${MY_MFU_INSTALL_PATH} \
-DWITH_LibCircle_PREFIX=${MY_MFU_INSTALL_PATH} \
-DCMAKE_INSTALL_PREFIX=${MY_MFU_INSTALL_PATH} &&
make -j8 install

And there is libgurt.so in ${MY_DAOS_INSTALL_PATH}/lib64/.
$ nm -D ${MY_DAOS_INSTALL_PATH}/lib64/libgurt.so | grep d_hash_rec_insert
000000000000fb40 T d_hash_rec_insert
0000000000010100 T d_hash_rec_insert_anonym

But looks like libgurt.so is not linked by libmfu.so.
$ ldd src/common/libmfu.so
linux-vdso.so.1 (0x00007ffda13f3000)
libmpi.so.40 => /lib/x86_64-linux-gnu/libmpi.so.40 (0x00007fd75c830000)
libdtcmp.so => /home/daos_agent/install/lib/libdtcmp.so (0x00007fd75c814000)
libcircle.so.2 => /home/daos_agent/install/lib/libcircle.so.2 (0x00007fd75c806000)
libbz2.so.1.0 => /lib/x86_64-linux-gnu/libbz2.so.1.0 (0x00007fd75c7f3000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fd75c601000)
libopen-rte.so.40 => /lib/x86_64-linux-gnu/libopen-rte.so.40 (0x00007fd75c545000)
libopen-pal.so.40 => /lib/x86_64-linux-gnu/libopen-pal.so.40 (0x00007fd75c497000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fd75c348000)
libhwloc.so.15 => /lib/x86_64-linux-gnu/libhwloc.so.15 (0x00007fd75c2f7000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fd75c2d4000)
liblwgrp.so => /home/daos_agent/install/lib/liblwgrp.so (0x00007fd75c2c5000)
/lib64/ld-linux-x86-64.so.2 (0x00007fd75c9b2000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fd75c2a7000)
libevent-2.1.so.7 => /lib/x86_64-linux-gnu/libevent-2.1.so.7 (0x00007fd75c251000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fd75c24b000)
libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007fd75c246000)
libevent_pthreads-2.1.so.7 => /lib/x86_64-linux-gnu/libevent_pthreads-2.1.so.7 (0x00007fd75c241000)
libudev.so.1 => /lib/x86_64-linux-gnu/libudev.so.1 (0x00007fd75c214000)
libltdl.so.7 => /lib/x86_64-linux-gnu/libltdl.so.7 (0x00007fd75c207000)"