|
Re: Timeouts/DAOS rendered useless when running IOR with SX/default object class
Hi Steffen,
Good catch! It sounds like we need to add a “LimitNOFILE” entry to our daos_server’s systemd unit file.
@Rosenzweig, Joel B could you please take of this? Thanks in advance.
Hi Steffen,
Good catch! It sounds like we need to add a “LimitNOFILE” entry to our daos_server’s systemd unit file.
@Rosenzweig, Joel B could you please take of this? Thanks in advance.
|
By
Johann
·
#1397
·
|
|
Re: Timeouts/DAOS rendered useless when running IOR with SX/default object class
A final "Hi" on that topic,
we have discovered the reason for the issue: The ulimit on the _server_ side was too low and it differs between regular users and daemons like the DAOS server. For the
A final "Hi" on that topic,
we have discovered the reason for the issue: The ulimit on the _server_ side was too low and it differs between regular users and daemons like the DAOS server. For the
|
By
Steffen Christgau
·
#1396
·
|
|
Re: Timeouts/DAOS rendered useless when running IOR with SX/default object class
Hi again once more,
meanwhile we checked the 'tcp' and the 'verbs' provider.
For 'tcp' we also experience the timeouts and an subsequently unusable DAOS system.
For 'verbs' (on an OmniPath network)
Hi again once more,
meanwhile we checked the 'tcp' and the 'verbs' provider.
For 'tcp' we also experience the timeouts and an subsequently unusable DAOS system.
For 'verbs' (on an OmniPath network)
|
By
Steffen Christgau
·
#1395
·
|
|
Re: Timeouts/DAOS rendered useless when running IOR with SX/default object class
Hi again,
Here is the output of a failed attempt to run IOR. It now crashed for 48 processes on a single client. For smaller process counts IOR succeeds with the same messages/warnings from
Hi again,
Here is the output of a failed attempt to run IOR. It now crashed for 48 processes on a single client. For smaller process counts IOR succeeds with the same messages/warnings from
|
By
Steffen Christgau
·
#1394
·
|
|
Re: Timeouts/DAOS rendered useless when running IOR with SX/default object class
Hi Alex,
Thanks for that input, we'll try to reproduce the issue with those settings and provide them ASAP.
Sure.
client $ ulimit -a
core file size (blocks, -c) 0
data seg size
Hi Alex,
Thanks for that input, we'll try to reproduce the issue with those settings and provide them ASAP.
Sure.
client $ ulimit -a
core file size (blocks, -c) 0
data seg size
|
By
Steffen Christgau
·
#1393
·
|
|
Re: Timeouts/DAOS rendered useless when running IOR with SX/default object class
Hi Steffen,
Could you enable OFI level logs by setting FI_LOG_LEVEL=warn on the client side and provide stdout/stderr output from runs that result in mercury erorrs/timeouts?
Also can you tell us
Hi Steffen,
Could you enable OFI level logs by setting FI_LOG_LEVEL=warn on the client side and provide stdout/stderr output from runs that result in mercury erorrs/timeouts?
Also can you tell us
|
By
Oganezov, Alexander A
·
#1392
·
|
|
Timeouts/DAOS rendered useless when running IOR with SX/default object class
Hi everybody,
during testing and performance assessment with IOR (latest Github version from main branch) we are facing problems with DAOS v1.1.3.
When running IOR from a single client node there is
Hi everybody,
during testing and performance assessment with IOR (latest Github version from main branch) we are facing problems with DAOS v1.1.3.
When running IOR from a single client node there is
|
By
Steffen Christgau
·
#1391
·
|
|
Re: Errors while compiling DAOS on ARM64 platform
Hi Huijun,
At one point in time, we added “// +build linux,amd64” to the netdetect.go file to enable it to build under ARM. Does your version of netdetect.go have the following at the end of
Hi Huijun,
At one point in time, we added “// +build linux,amd64” to the netdetect.go file to enable it to build under ARM. Does your version of netdetect.go have the following at the end of
|
By
Rosenzweig, Joel B <joel.b.rosenzweig@...>
·
#1390
·
|
|
Errors while compiling DAOS on ARM64 platform
Hi all,
I am trying to compile DAOS on ARM64 platform (little endian). I am working with the branch'tanabarr/control-no-ipmctl-May2020' to avoid the ipmctl dependency.
However, I got errors below
Hi all,
I am trying to compile DAOS on ARM64 platform (little endian). I am working with the branch'tanabarr/control-no-ipmctl-May2020' to avoid the ipmctl dependency.
However, I got errors below
|
By
Wu Huijun
·
#1389
·
|
|
Re: Questions about Daos consistency
T3 starts before T1, so T3 can obtain a timestamp less than T1. T1 has not yet started when T3 is read, so there will be no uncertain reading
When T1 writes C1, it does not check the read timestamp
T3 starts before T1, so T3 can obtain a timestamp less than T1. T1 has not yet started when T3 is read, so there will be no uncertain reading
When T1 writes C1, it does not check the read timestamp
|
By
段世博
·
#1388
·
|
|
Re: Questions about Daos consistency
I may be missing something here but assuming T3 is at a later timestamp to T1, the read of C1 would update the read timestamp in the negative entry for C1 (based on a hash of the key). Before T1
I may be missing something here but assuming T3 is at a later timestamp to T1, the read of C1 would update the read timestamp in the negative entry for C1 (based on a hash of the key). Before T1
|
By
Olivier, Jeffrey V
·
#1387
·
|
|
Re: Questions about Daos consistency
You are right. This can also happen with DAOS. I will correct that document.
Thanks,
liwei
You are right. This can also happen with DAOS. I will correct that document.
Thanks,
liwei
|
By
Li, Wei G
·
#1386
·
|
|
Re: Questions about Daos consistency
I found that the concurrency control of DAOS is similar to CockroachDB, but the following situations may occur in CockroachDB according to jepsen analysis
I found that the concurrency control of DAOS is similar to CockroachDB, but the following situations may occur in CockroachDB according to jepsen analysis
|
By
段世博
·
#1385
·
|
|
DFS fio engine
Hi there,
I just would like to share with you that the DAOS File System (DFS) engine has been integrated into the upstream FIO repository (https://github.com/axboe/fio).
How to build it on
Hi there,
I just would like to share with you that the DAOS File System (DFS) engine has been integrated into the upstream FIO repository (https://github.com/axboe/fio).
How to build it on
|
By
Johann
·
#1384
·
|
|
Re: Questions about Daos consistency
Yes. A DAOS client can only "see a state” via unversioned transactions (including I/O operations submitted without an explicit transaction) and explicitly-created snapshots. If an application hacks
Yes. A DAOS client can only "see a state” via unversioned transactions (including I/O operations submitted without an explicit transaction) and explicitly-created snapshots. If an application hacks
|
By
Li, Wei G
·
#1383
·
|
|
Questions about Daos consistency
In the VOS document, the MVCC section mentions "The MVCC rules ensure that transactions execute as if they are serialized in their epoch order while complying with external consistency, as long as the
In the VOS document, the MVCC section mentions "The MVCC rules ensure that transactions execute as if they are serialized in their epoch order while complying with external consistency, as long as the
|
By
段世博
·
#1382
·
|
|
Questions about Daos consistency
In the VOS document, the MVCC section mentions "The MVCC rules ensure that transactions execute as if they are serialized in their epoch order while complying with external consistency, as long as the
In the VOS document, the MVCC section mentions "The MVCC rules ensure that transactions execute as if they are serialized in their epoch order while complying with external consistency, as long as the
|
By
段世博
·
#1381
·
|
|
Re: [EXTERNAL SENDER] Re: [daos] Startup Errors
Hello Neale,
I’m happy to work with you directly on this to get you past any hurdles if you would like, my e-mail istom.nabarro@....
The TRANSIENT_FAILURE does indicate some local network
Hello Neale,
I’m happy to work with you directly on this to get you past any hurdles if you would like, my e-mail istom.nabarro@....
The TRANSIENT_FAILURE does indicate some local network
|
By
Nabarro, Tom
·
#1380
·
|
|
Re: [EXTERNAL SENDER] Re: [daos] Startup Errors
Hi Tom,
I tried your suggestion by uninstalling / reinstalling DAOS RPMs to get a blank config file then added things line by line. Unfortunately, I ended up getting "insufficient information in
Hi Tom,
I tried your suggestion by uninstalling / reinstalling DAOS RPMs to get a blank config file then added things line by line. Unfortunately, I ended up getting "insufficient information in
|
By
Neale Petrillo (Contractor)
·
#1379
·
|
|
Re: [EXTERNAL SENDER] Re: [daos] Startup Errors
I think getting the most basic configuration working is probably the best way forward given that dmg is not connecting, try with an empty config file (discovery mode) on a single host and on that same
I think getting the most basic configuration working is probably the best way forward given that dmg is not connecting, try with an empty config file (discovery mode) on a single host and on that same
|
By
Nabarro, Tom
·
#1378
·
|