Re: DAOS_test failed


Lombardi, Johann
 

Hi Anton,

 

Are you able to create files manually under the dfuse mountpoint? Does ior work with file per process (-F)?

 

Cheers,

Johann

 

From: <daos@daos.groups.io> on behalf of "anton.brekhov@..." <anton.brekhov@...>
Reply-To: "daos@daos.groups.io" <daos@daos.groups.io>
Date: Sunday 20 September 2020 at 20:17
To: "daos@daos.groups.io" <daos@daos.groups.io>
Subject: Re: [daos] DAOS_test failed

 

Hi everyone! 

I tried to launch ior test on client node and got the error:

[root@sky08 ~]# mpirun --allow-run-as-root  -np 32  ior -a POSIX -b 1G -w -r  -o /mnt/dfuse/testFile_POSIX -t 128k -k -G 123

IOR-3.4.0+dev: MPI Coordinated Test of Parallel I/O

Began               : Sun Sep 20 21:16:24 2020

Command line        : ior -a POSIX -b 1G -w -r -o /mnt/dfuse/testFile_POSIX -t 128k -k -G 123

Machine             : Linux sky08

TestID              : 0

StartTime           : Sun Sep 20 21:16:24 2020

WARNING: Backend returned error during statfs.

POSIX couldn't call statvfs: No such file or directory

 

Options:

api                 : POSIX

apiVersion          :

test filename       : /mnt/dfuse/testFile_POSIX

access              : single-shared-file

type                : independent

segments            : 1

ordering in a file  : sequential
ordering inter file : no tasks offsets

nodes               : 1

tasks               : 32

clients per node    : 32

repetitions         : 1

xfersize            : 131072 bytes

blocksize           : 1 GiB

aggregate filesize  : 32 GiB

 

Results:

 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter

------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----

--------------------------------------------------------------------------

MPI_ABORT was invoked on rank 30 in communicator MPI_COMM_WORLD

with errorcode -1.

 

NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.

You may or may not see output from other processes, depending on

exactly when Open MPI kills them.

--------------------------------------------------------------------------

ERROR: open64("/mnt/dfuse/testFile_POSIX", 66, 0664) failed, (aiori-POSIX.c:440)

ERROR: open64("/mnt/dfuse/testFile_POSIX", 66, 0664) failed, (aiori-POSIX.c:440)

ERROR: open64("/mnt/dfuse/testFile_POSIX", 66, 0664) failed, (aiori-POSIX.c:440)

ERROR: open64("/mnt/dfuse/testFile_POSIX", 66, 0664) failed, (aiori-POSIX.c:440)

ERROR: open64("/mnt/dfuse/testFile_POSIX", 66, 0664) failed, (aiori-POSIX.c:440)

ERROR: open64("/mnt/dfuse/testFile_POSIX", 66, 0664) failed, (aiori-POSIX.c:440)

ERROR: open64("/mnt/dfuse/testFile_POSIX", 66, 0664) failed, (aiori-POSIX.c:440)

ERROR: open64("/mnt/dfuse/testFile_POSIX", 66, 0664) failed, (aiori-POSIX.c:440)

ERROR: open64("/mnt/dfuse/testFile_POSIX", 66, 0664) failed, (aiori-POSIX.c:440)

ERROR: open64("/mnt/dfuse/testFile_POSIX", 66, 0664) failed, (aiori-POSIX.c:440)

ERROR: open64("/mnt/dfuse/testFile_POSIX", 66, 0664) failed, (aiori-POSIX.c:440)

ERROR: open64("/mnt/dfuse/testFile_POSIX", 66, 0664) failed, (aiori-POSIX.c:440)

ERROR: open64("/mnt/dfuse/testFile_POSIX", 66, 0664) failed, (aiori-POSIX.c:440)

ERROR: open64("/mnt/dfuse/testFile_POSIX", 66, 0664) failed, (aiori-POSIX.c:440)

ERROR: open64("/mnt/dfuse/testFile_POSIX", 66, 0664) failed, (aiori-POSIX.c:440)

ERROR: open64("/mnt/dfuse/testFile_POSIX", 66, 0664) failed, (aiori-POSIX.c:440)

ERROR: open64("/mnt/dfuse/testFile_POSIX", 66, 0664) failed, (aiori-POSIX.c:440)

ERROR: open64("/mnt/dfuse/testFile_POSIX", 66, 0664) failed, (aiori-POSIX.c:440)

ERROR: open64("/mnt/dfuse/testFile_POSIX", 66, 0664) failed, (aiori-POSIX.c:440)

ERROR: open64("/mnt/dfuse/testFile_POSIX", 66, 0664) failed, (aiori-POSIX.c:440)

ERROR: open64("/mnt/dfuse/testFile_POSIX", 66, 0664) failed, (aiori-POSIX.c:440)

ERROR: open64("/mnt/dfuse/testFile_POSIX", 66, 0664) failed, (aiori-POSIX.c:440)

ERROR: open64("/mnt/dfuse/testFile_POSIX", 66, 0664) failed, (aiori-POSIX.c:440)

ERROR: open64("/mnt/dfuse/testFile_POSIX", 66, 0664) failed, (aiori-POSIX.c:440)

ERROR: open64("/mnt/dfuse/testFile_POSIX", 66, 0664) failed, (aiori-POSIX.c:440)

ERROR: open64("/mnt/dfuse/testFile_POSIX", 66, 0664) failed, (aiori-POSIX.c:440)

ERROR: open64("/mnt/dfuse/testFile_POSIX", 66, 0664) failed, (aiori-POSIX.c:440)

ERROR: open64("/mnt/dfuse/testFile_POSIX", 66, 0664) failed, (aiori-POSIX.c:440)

ERROR: open64("/mnt/dfuse/testFile_POSIX", 66, 0664) failed, (aiori-POSIX.c:440)

ERROR: open64("/mnt/dfuse/testFile_POSIX", 66, 0664) failed, (aiori-POSIX.c:440)

ERROR: open64("/mnt/dfuse/testFile_POSIX", 66, 0664) failed, (aiori-POSIX.c:440)

ERROR: open64("/mnt/dfuse/testFile_POSIX", 66, 0664) failed, (aiori-POSIX.c:440)

[sky08:04378] PMIX ERROR: UNREACHABLE in file server/pmix_server.c at line 2193

[sky08:04378] PMIX ERROR: UNREACHABLE in file server/pmix_server.c at line 2193

[sky08:04378] PMIX ERROR: UNREACHABLE in file server/pmix_server.c at line 2193

[sky08:04378] PMIX ERROR: UNREACHABLE in file server/pmix_server.c at line 2193

[sky08:04378] PMIX ERROR: UNREACHABLE in file server/pmix_server.c at line 2193

[sky08:04378] 31 more processes have sent help message help-mpi-api.txt / mpi-abort

[sky08:04378] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages

What can it be?

---------------------------------------------------------------------
Intel Corporation SAS (French simplified joint stock company)
Registered headquarters: "Les Montalets"- 2, rue de Paris,
92196 Meudon Cedex, France
Registration Number:  302 456 199 R.C.S. NANTERRE
Capital: 4,572,000 Euros

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

Join daos@daos.groups.io to automatically receive all group messages.