daos@daos.groups.io

Re: Jenkins test

dongfeier

Hello,

useradd: UID 0 is not unique
The command '/bin/sh -c useradd --no-log-init --uid $UID --user-group --create-home --shell /bin/bash             --home /home/daos daos_server' returned a non-zero code: 4

Jenkins not configured to notify users of failed builds.
Scripts not permitted to use staticMethod org.codehaus.groovy.runtime.DefaultGroovyMethods getAt java.lang.Object java.lang.String. Administrators can decide whether to approve or reject this signature.
Error when executing unsuccessful post condition:
org.jenkinsci.plugins.scriptsecurity.sandbox.RejectedAccessException: Scripts not permitted to use staticMethod org.codehaus.groovy.runtime.DefaultGroovyMethods getAt java.lang.Object java.lang.String
	at org.jenkinsci.plugins.scriptsecurity.sandbox.whitelists.StaticWhitelist.rejectStaticMethod(StaticWhitelist.java:279)
	at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onGetArray(SandboxInterceptor.java:476)
	at org.kohsuke.groovy.sandbox.impl.Checker$11.call(Checker.java:484)
	at org.kohsuke.groovy.sandbox.impl.Checker.checkedGetArray(Checker.java:489)
	at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.getArray(SandboxInvoker.java:45)
	at com.cloudbees.groovy.cps.impl.ArrayAccessBlock.rawGet(ArrayAccessBlock.java:21)
	at notifyBrokenBranch.call(notifyBrokenBranch.groovy:37)
	at WorkflowScript.run(WorkflowScript:1049)
	at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.delegateAndExecute(ModelInterpreter.groovy:137)
	at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.runPostConditions(ModelInterpreter.groovy:761)
	at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.catchRequiredContextForNode(ModelInterpreter.groovy:395)
	at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.catchRequiredContextForNode(ModelInterpreter.groovy:393)
	at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.runPostConditions(ModelInterpreter.groovy:760)
	at com.cloudbees.groovy.cps.CpsDefaultGroovyMethods.each(CpsDefaultGroovyMethods:2030)
	at com.cloudbees.groovy.cps.CpsDefaultGroovyMethods.each(CpsDefaultGroovyMethods:2015)
	at com.cloudbees.groovy.cps.CpsDefaultGroovyMethods.each(CpsDefaultGroovyMethods:2056)
	at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.runPostConditions(ModelInterpreter.groovy:750)
	at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.runPostConditions(ModelInterpreter.groovy)
	at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.executePostBuild(ModelInterpreter.groovy:728)
	at ___cps.transform___(Native Method)
	at com.cloudbees.groovy.cps.impl.PropertyishBlock$ContinuationImpl.get(PropertyishBlock.java:74)
	at com.cloudbees.groovy.cps.LValueBlock$GetAdapter.receive(LValueBlock.java:30)
	at com.cloudbees.groovy.cps.impl.PropertyishBlock$ContinuationImpl.fixName(PropertyishBlock.java:66)
	at sun.reflect.GeneratedMethodAccessor430.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
	at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21)
	at com.cloudbees.groovy.cps.Next.step(Next.java:83)
	at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174)
	at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163)
	at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:129)
	at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:268)
	at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163)
	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18)
	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:51)
	at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:185)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:402)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$400(CpsThreadGroup.java:96)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:314)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:278)
	at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:67)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:139)
	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
	at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

I have found the cause of the problem because the uid is repeated when adding users. Thank you very much

DAOS Community Update / Mar'22

Lombardi, Johann

#1559

Hi there,

Please find below the DAOS community newsletter for March 2022.

Past Events (February)

Dell Technologies HPC Community Event (February 9^th)
DAOS: High Performance Storage for All HPC Workloads and Scales
Kelsey Prantis (Intel)
https://www.dellhpc.org/event_2-9-2022.html

Upcoming Events

2022 Energy HPC Conference (March 3^rd)
DAOS Use at Argonne
Kevin Harms (Argonne)
Johann Lombardi (Intel)
Mohamad Chaarawi (Intel)
https://energyhpc.rice.edu/program/

Release

Current stable release is 2.0.1 and 2.0.2 is under development.
Master is the development branch for the future 2.2 release.
Major recent 2.0 changes to be available in 2.0.2:

Build go code shipped in RPM with go >= 1.17
Fix a bug with key enumeration and multiple shards
Many documentation fixes (e.g. fix broken links)
Use larger ABT stack for NVMe poll ULT
Reject integer keys that are smaller than 8 bytes
Update the C-based raft implementation to pick a membership fix
Disable dfuse-level readahead when FUSE is configured in writeback mode and already drive readahead from the kernel.

Major recent master changes:

Fix a problem between mercury and libfabric causing some bulk transfers to be stuck indefinitely
Reduce CPU load caused by the engine when DAOS system is idle
Add support for owner/group to libdfs
Automatically calculate the required number of huge pages
Several VMD and SSD hotplug fixes
Document the need to disable shared received context for Intel E810-C cards
Rewrite of the netdetect module
Add UCX support to the manual (non RPM) build infrastructure

What is coming:

2.0.2 validation
2.2.0 feature freeze

R&D

Major features under development:

Checksum scrubbing
UCX support
LDMS plugin to export DAOS metrics (targeted for 2.2)
API to collect libdaos metrics to be integrated with Darshan (targeted for 2.4)
Multi-user dfuse (targeted for 2.4)
More aggressive caching in dfuse for AI APPs (targeted for 2.4)
TensorFlow-IO plugin for DFS

https://github.com/daos-stack/tensorflow-io-daos/blob/devel/docs/daos_tf_docs.md

Pathfinding:

MariaDB DAOS engine with predicate pushdown to the DAOS storage nodes

Prototyped DAOS MariaDB engine available here: https://github.com/daos-stack/mariadb
PR for pipeline API (#6238)
Work in progress to support pipeline API in the engine

Leveraging the Intel Data Streaming Accelerator (DSA) to accelerate DAOS

Prototype leveraging DSA for VOS aggregation delivered

S3 support via a DAOS backend to Rados Gateway (RGW)
GPU data path optimizations

News

Sodacode 2022 hackathon started today with participation of the DAOS team.
The list of DAOS activities offered in this hackathon has been created in jira: https://daosio.atlassian.net/browse/DAOS-10010?jql=labels%20%3D%20%22SODACODE2022%22%20and%20statusCategory%20!%3D%20Done

See https://events.linuxfoundation.org/sodacode/ for more information.

---------------------------------------------------------------------
Intel Corporation SAS (French simplified joint stock company)
Registered headquarters: "Les Montalets"- 2, rue de Paris,
92196 Meudon Cedex, France
Registration Number: 302 456 199 R.C.S. NANTERRE
Capital: 4,572,000 Euros

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

Re: Jenkins test

Murrell, Brian

#1558

					On Wed, 2022-03-02 at 01:38 -0800, dongfeier wrote:
Hello,
After the Jenkins environment configuration is completed, the test is
carried out, and the following exceptions are foundSome function needs to be moved to the trusted library.  There is not
enough information in your screenshot to know which function though. 
For this reason copy and paste of text is preferred to screenshots as
it typically carries much more information, particularly in the case of
java stacktraces.

Cheers,
b.
				

Re: Jenkins test

dongfeier

#1557

Hello,
After the Jenkins environment configuration is completed, the test is carried out, and the following exceptions are found

Please let me know if you have any suggestions.
Thanks in advance

Re: Jenkins test

Murrell, Brian

#1556

					On Mon, 2022-02-28 at 17:45 -0800, 15735154041@... wrote:
Hello,Hi,

Jenkins test uses the pipeline shared library. After the
configuration is completed, there is the following exception "no such
DSL method 'commitpragmatrusted' found among steps". Please let me
know if you have any suggestions.That is implemented in:
https://github.com/daos-stack/trusted-pipeline-lib/blob/master/vars/commitPragmaTrusted.groovy

which is supplied by the:

https://github.com/daos-stack/trusted-pipeline-lib

hared library.  You need to apply that library as a trusted library to
your Jenkins configuration as instructed at:

https://www.jenkins.io/doc/book/pipeline/shared-libraries/#global-shared-libraries

Cheers,
b.
				

Re: Jenkins test

dongfeier

#1555

Hello,
Jenkins test uses the pipeline shared library. After the configuration is completed, there is the following exception "no such DSL method 'commitpragmatrusted' found among steps". Please let me know if you have any suggestions.
Thanks in advance

Re: Jenkins test

Lombardi, Johann

#1554

Hi there,

Please find below the plugin usage page from Jenkins. HTH.

Cheers,
Johann

Warning: Scripted pipeline jobs are not detected by this plugin. Support for declarative pipeline jobs is experimental. Number of Jobs: 46

Jobs by Plugin

Plugin Name	Version	Number of Jobs	Jobs	Has Dependants?
AnsiColor	1.0.0	0		false
Ant Plugin	1.11	0		true
Build Blocker Plugin	1.7.7	12		false
Build Failure Analyzer	2.0.1	968		true
Build Name and Description Setter	2.2.0	0		false
Build Pipeline Plugin	1.5.8	0		false
Build Timeout	1.20	43		false
Checks API plugin	1.7.2	0		true
Code Coverage API Plugin	1.4.1	0		true
Conditional BuildStep	1.4.1	0		true
Config File Provider Plugin	3.8.1	0		true
Confluence Publisher	2.0.6	0		false
Copy Artifact Plugin	1.46.1	0		true
Credentials Binding Plugin	1.27	1		true
Docker Pipeline	1.26	0		false
Email Extension Plugin	2.83	0		true
Email Extension Template Plugin	1.2	0		false
Environment Injector Plugin	2.4.0	0		true
File Operations Plugin	1.11	0		false
Folders Plugin	6.16	0		true
Forensics API Plugin	1.3.0	0		true
Git Forensics Plugin	1.2.0	0		false
GitHub plugin	1.34.1	0		true
GitLab Plugin	1.5.20	16		false
Gradle Plugin	1.37.1	0		true
Groovy	2.4	0		false
HTML Publisher plugin	1.25	0		true
HTTP Request Plugin	1.11	0		false
Ivy Plugin	2.1	0		true
JUnit Plugin	1.53	0		true
Javadoc Plugin	1.6	0		true
Jenkins Artifactory Plugin	3.14.1	0		false
Jenkins BuildResultTrigger Plug-in	0.17	0		false
Jenkins CVS Plug-in	2.19	0		true
Jenkins Clover plugin	4.12.1	0		false
Jenkins Cobertura Plugin	1.16	0		false
Jenkins Cppcheck Plug-in	1.25	0		false
Jenkins Doxygen Plug-in	0.18	0		false
Jenkins Git plugin	4.8.2	2		true
Jenkins Jira plugin	3.5	7		true
Jenkins Job Configuration History Plugin	2.28.1	3		true
Jenkins Libvirt Agents plugin	1.9.2	12		false
Jenkins Mailer Plugin	1.34	0		true
Jenkins Mercurial plugin	2.15	0		false
Jenkins Multijob plugin	1.36	0		true
Jenkins Parameterized Trigger plugin	2.41	0		true
Jenkins PostBuildScript Plugin	3.0.0	0		false
Jenkins SSH plugin	2.6.1	0		false
Jenkins Subversion Plug-in	2.14.4	0		true
Jenkins Throttle Concurrent Builds Plug-in	2.4	12		false
Jenkins Valgrind Plug-in	0.28	0		false
Jenkins Workspace Cleanup Plugin	0.39	0		false
Job and Node ownership plugin	0.13.0	689		false
Lockable Resources plugin	2.11	0		true
Managed Scripts	1.5.4	0		false
Mask Passwords Plugin	3.0	0		false
Matrix Authorization Strategy Plugin	2.6.8	8		true
Maven Integration plugin	3.12	0		true
Pipeline GitHub Notify Step Plugin	1.0.5	0		false
Pipeline Utility Steps	2.10.0	0		false
Pipeline: Basic Steps	2.24	0		true
Pipeline: Build Step	2.15	0		true
Pipeline: Declarative	1.9.1	431		true
Pipeline: GitHub	2.7	0		false
Pipeline: Groovy	2.93	0		true
Pipeline: Input Step	2.12	0		true
Pipeline: Job	2.41	410		true
Pipeline: Milestone Step	1.3.2	0		true
Pipeline: Multibranch	2.26	923		true
Pipeline: Nodes and Processes	2.39	0		true
Pipeline: SCM Step	2.13	0		true
Pipeline: Shared Groovy Libraries	2.21	0		true
Pipeline: Stage Step	2.5	0		true
Plot plugin	2.1.9	0		false
Post build task	1.9	0		false
Publish Over SSH	1.22	0		false
Rebuilder	1.32	12		true
SSH Agent Plugin	1.23	2		false
SSH Pipeline Steps	2.0.0	0		false
Timestamper	1.13	45		false
Token Macro Plugin	266.v44a80cf277fd	0		true
Warnings Next Generation Plugin	9.5.0	0		false
xUnit plugin	3.0.2	0		false

Other Plugins

Plugin Name	Version	Has Dependants?
Active Choices Plug-in	2.5.6	false
Analysis Model API Plugin	10.3.0	true
Audit Trail	3.10	false
Authentication Tokens API Plugin	1.4	true
Autofavorite for Blue Ocean	1.2.4	true
Basic Branch Build Strategies Plugin	1.3.2	false
Bitbucket Branch Source Plugin	2.9.11	true
Bitbucket Pipeline for Blue Ocean	1.24.8	true
Blue Ocean	1.24.8	false
Blue Ocean Core JS	1.24.8	true
Blue Ocean Pipeline Editor	1.24.8	true
Bootstrap 4 API Plugin	4.6.0-3	true
Bootstrap 5 API Plugin	5.1.0-3	true
Branch API Plugin	2.6.5	true
Build Blocker Plugin	1.7.7	false
Build Failure Analyzer	2.0.1	true
Build History Metrics plugin	93.v0e17adc6ccea	false
Build Monitor View	1.12+build.201809061734	false
Caffeine API Plugin	2.9.2-29.v717aac953ff3	true
Command Agent Launcher Plugin	1.6	true
Common API for Blue Ocean	1.24.8	true
Config API for Blue Ocean	1.24.8	true
Config AutoRefresh Plugin	1.0	false
Credentials Plugin	2.5	true
DTKit 2 API.	3.0.0	true
Dashboard for Blue Ocean	1.24.8	true
DataTables.net API Plugin	1.10.25-3	true
Disable GitHub Multibranch Status Plugin	1.2	false
Display URL API	2.3.5	true
Display URL for Blue Ocean	2.4.1	true
Docker Commons Plugin	1.17	true
Durable Task Plugin	1.39	true
ECharts API Plugin	5.1.2-11	true
EZ Templates	1.3.4	false
EnvInject API Plugin	1.7	true
Events API for Blue Ocean	1.24.8	true
External Monitor Job Type Plugin	1.7	true
Favorite	2.3.3	true
Font Awesome API Plugin	5.15.4-1	true
Git Parameter Plug-In	0.9.13	false
Git Pipeline for Blue Ocean	1.24.8	true
GitHub API Plugin	1.123	true
GitHub Branch Source Plugin	2.11.2	true
GitHub Pipeline for Blue Ocean	1.24.8	true
Green Balls	1.15.1	false
Handy Uri Templates 2.x API Plugin	2.1.8-1.0	true
Hudson Personal View	1.8	false
Hudson global-build-stats plugin	1.5	true
Infrastructure plugin for Publish Over X	0.22	true
JAXB plugin	2.3.0.1	true
JIRA Integration for Blue Ocean	1.24.8	true
JQuery3 API Plugin	3.6.0-2	true
JWT for Blue Ocean	1.24.8	true
Jackson 2 API Plugin	2.12.4	true
Java JSON Web Token (JJWT) Plugin	0.11.2-9.c8b45b8bb173	true
JavaScript GUI Lib: ACE Editor bundle plugin	1.1	true
JavaScript GUI Lib: Handlebars bundle plugin	3.0.8	true
JavaScript GUI Lib: Moment.js bundle plugin	1.1.1	true
JavaScript GUI Lib: jQuery bundles (jQuery and jQuery UI) plugin	1.2.1	true
Jenkins Apache HttpComponents Client 4.x API Plugin	4.5.13-1.0	true
Jenkins Design Language	1.24.8	true
Jenkins Extended Read Permission Plugin	3.2	false
Jenkins GIT server Plugin	1.10	true
Jenkins Git client plugin	3.9.0	true
Jenkins JSch dependency plugin	0.1.55.2	true
Jenkins Job Configuration History Plugin	2.28.1	true
Jenkins Libvirt Agents plugin	1.9.2	false
Jenkins Pipeline Remote Loader Plugin	1.5	false
Jenkins Platform Labeler Plugin	987.v674996136faf	false
Jenkins Priority Sorter Plugin	4.0.0	false
Jenkins Pub-Sub "light" Bus	1.16	true
Jenkins Slave SetupPlugin	1.10	false
JenkinsLint Plugin	0.14.0	false
Job Import Plugin	3.4	false
LDAP Plugin	2.7	true
Lenient shutdown plugin	1.1.1	false
Mail Watcher Plugin	1.16	false
MapDB API Plugin	1.0.9.0	true
Matrix Authorization Strategy Plugin	2.6.8	true
Matrix Project Plugin	1.19	true
Matrix Reloaded Plugin	1.1.3	false
Metrics Plugin	4.0.2.8	true
Monitoring	1.88.0	false
Next Build Number Plugin	1.6	false
OWASP Markup Formatter Plugin	2.1	true
OkHttp Plugin	3.14.9	true
Oracle Java SE Development Kit Installer Plugin	1.5	true
PAM Authentication plugin	1.6	true
Persistent Parameter Plugin	1.3	false
Personalization for Blue Ocean	1.24.8	true
Pipeline	2.6	true
Pipeline Graph Analysis Plugin	1.11	true
Pipeline SCM API for Blue Ocean	1.24.8	true
Pipeline implementation for Blue Ocean	1.24.8	true
Pipeline: API	2.46	true
Pipeline: Declarative Extension Points API	1.9.1	true
Pipeline: GitHub Groovy Libraries	1.0	false
Pipeline: Job	2.41	true
Pipeline: Model API	1.9.1	true
Pipeline: REST API Plugin	2.19	true
Pipeline: Stage Tags Metadata	1.9.1	true
Pipeline: Stage View Plugin	2.19	true
Pipeline: Step API	2.24	true
Pipeline: Supporting APIs	3.8	true
Plain Credentials Plugin	1.7	true
Plugin Usage - Plugin	1.2	false
Plugin Utilities API Plugin	2.4.0	true
Popper.js 2 API Plugin	2.9.3-1	true
Popper.js API Plugin	1.16.1-2	true
Prometheus metrics plugin	2.0.10	false
REST API for Blue Ocean	1.24.8	true
REST Implementation for Blue Ocean	1.24.8	true
Rebuilder	1.32	true
Resource Disposer Plugin	0.16	true
Role-based Authorization Strategy	3.2.0	true
Run Condition Plugin	1.5	true
SCM API Plugin	2.6.5	true
SSH Build Agents plugin	1.33.0	false
SSH Credentials Plugin	1.19	true
SSH server	3.1.0	true
Script Security Plugin	1.78	true
Server Sent Events (SSE) Gateway Plugin	1.24	true
Sidebar Link	1.12.0	false
Simple Queue Plugin	1.4.1	false
Snakeyaml API Plugin	1.29.1	true
Strict Crumb Issuer Plugin	2.1.0	false
Structs Plugin	1.23	true
Test Results Analyzer Plugin	0.3.5	false
ThinBackup	1.10	false
Trilead API Plugin	1.0.13	true
Variant Plugin	1.4	true
View Job Filters	2.3	true
WMI Windows Agents Plugin	1.8	true
Web for Blue Ocean	1.24.8	true
bouncycastle API Plugin	2.24	true
build-metrics	1.3	false
built-on-column	1.1	true
excludeMatrixParent	1.1	false
i18n for Blue Ocean	1.24.8	true
jQuery plugin	1.12.4-1	true

From: <daos@daos.groups.io> on behalf of "15735154041@..." <15735154041@...>
Reply-To: "daos@daos.groups.io" <daos@daos.groups.io>
Date: Monday 28 February 2022 at 06:32
To: "daos@daos.groups.io" <daos@daos.groups.io>
Subject: [daos] Jenkins test

Hello everyone:
Recently, when I deployed the Jenkins test environment, a similar problem occurred after the task was executed: "no such DSL method 'valgrindreportpublish' found among steps". I look forward to providing a list of Jenkins test dependent plug-ins.
Best regards!

---------------------------------------------------------------------
Intel Corporation SAS (French simplified joint stock company)
Registered headquarters: "Les Montalets"- 2, rue de Paris,
92196 Meudon Cedex, France
Registration Number: 302 456 199 R.C.S. NANTERRE
Capital: 4,572,000 Euros

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

Jenkins test

dongfeier

#1553

Hello everyone:
Recently, when I deployed the Jenkins test environment, a similar problem occurred after the task was executed: "no such DSL method 'valgrindreportpublish' found among steps". I look forward to providing a list of Jenkins test dependent plug-ins.
Best regards!

Re: 'access_points' must contain resolvable addresses complaint

Hannappel, Juergen

#1552

The config file was created by modifying the file that was installed from the RPM, what I showd in the mail were just the lines which are not commented out.
I commented out the bdev_include, but that did not help.

See here the comple diff to the version in utils/config (why is that one different from the one installed initailly in /etc/daos ?):
[root@asapo-srv09 ~]# diff /usr/local/src/daos/utils/config/daos_server.yml /etc/daos/daos_server.yml
15c15
< #name: daos_server
---
> name: daos_server
27c27
< #access_points: ['hostname1']
---
> access_points: ['131.169.183.155']
75,76c75,76
< ## ofi+verbs for Infiniband/RoCE and
< ## ofi+tcp for non-RDMA-capable Ethernet.
---
> ## ofi+verbs;ofi_rxm for Infiniband/RoCE and
> ## ofi+sockets for non-RDMA-capable Ethernet.
78c78
< #provider: ofi+verbs
---
> provider: ofi+verbs;ofi_rxm
94,99d93
< ## CART: Disable SRX
< ## parameters shared with client.
< #
< #disable_srx: false
< #
< #
107c101
< #bdev_include: ["0000:81:00.1","0000:81:00.2","0000:81:00.3"]
---
> #bdev_include: ["01:00.0","02:00.0"]
176c170
< ## Number of hugepages to allocate for DMA buffer memory
---
> ## Number of hugepages to allocate for use by NVMe SSDs
179,189c173,175
< ## through SPDK. Note that each target requires 1 GB of hugepage space, so
< ## this value needs to represent the total amount of hugespace required for
< ## all targets across all engines on host, divided by the system hugepage size.
< ## If not set here, an appropriate value will be automatically calculated and
< ## default system hugepage size will be used.
< #
< ## Example: (2 engines * (8 targets/engine * 1GB)) / 2MB hugepage size = 16834
< #
< ## Hugepages are mandatory with NVME SSDs configured and optional without.
< ## To disabled the use of hugepages when no NVMe SSDs are configured, set
< ## nr_hugepages to -1.
---
> ## through SPDK. This indicates the number to be used for each spawned
> ## I/O Engines, so the total will be this number * number of I/O Engines.
> ## Default system hugepage size will be used.
191c177,178
< ##nr_hugepages: -1
---
> ## default: 4096
> #nr_hugepages: 4096
353,354c340
< #    bdev_list: ["0000:81:00.0", "0000:82:00.0"] # generate regular nvme.conf
< #
---
> #    bdev_list: ["0000:81:00.0"] # generate regular nvme.conf
359,365d344
< #
< #    # Optional override, will be automatically generated based on NUMA affinity.
< #    # Filter hot-pluggable devices by PCI bus-ID by specifying a hexadecimal
< #    # range. Hotplug events relating to devices with PCI bus-IDs outside this range
< #    # will not be processed by this engine. Empty or unset range signifies allow all.
< #    bdev_busid_range: 0x80-0x8f
< #    #bdev_busid_range: 128-143
501,512c480,481
< #    #class: nvme
< #    #bdev_list: ["0000:5d:05.5"]
< #
< #    #class: nvme
< #    #bdev_list: ["0000:da:00.0", "0000:db:00.0"] # generate regular nvme.conf
< #
< #    # Optional override, will be automatically generated based on NUMA affinity.
< #    # Filter hot-pluggable devices by PCI bus-ID by specifying a hexadecimal
< #    # range. Hotplug events relating to devices with PCI bus-IDs outside this range
< #    # will not be processed by this engine. Empty or unset range signifies allow all.
< #    #bdev_busid_range: 0xd0-0xdf
< #    #bdev_busid_range: 208-223
---
> #    class: nvme
> #    bdev_list: ["0000:82:00.0","0000:5d:05.5"]

Re: 'access_points' must contain resolvable addresses complaint

Nabarro, Tom

#1551

Hello,

Could you please try to use one of the examples in utils/config/examples and look at the reference documentation in utils/config/daos_server.yml. The config file you are using below is not valid. Specifically bdev_include is not a global parameter.

Regards,

Tom

From: daos@daos.groups.io <daos@daos.groups.io> On Behalf Of Hannappel, Juergen
Sent: Wednesday, February 16, 2022 3:56 PM
To: daos@daos.groups.io
Subject: Re: [daos] 'access_points' must contain resolvable addresses complaint

Hi,
there is no log file yet, all entries are in the journal.
They look like this:
Feb 16 16:51:32 asapo-srv09.desy.de systemd[1]: Started DAOS Server.
Feb 16 16:51:32 asapo-srv09.desy.de daos_server[267723]: DAOS Server config loaded from /etc/daos/daos_server.yml
Feb 16 16:51:32 asapo-srv09.desy.de daos_server[267723]: no control log file specified; logging to stdout
Feb 16 16:51:32 asapo-srv09.desy.de daos_server[267723]: No DAOS I/O Engines in configuration, DAOS Control Server starting in discovery mode
Feb 16 16:51:32 asapo-srv09.desy.de daos_server[267723]: ERROR: retrieve replicas from config: serverconfig: code = 705 description = "invalid list of access points in configuration"
Feb 16 16:51:32 asapo-srv09.desy.de daos_server[267723]: ERROR: serverconfig: code = 705 resolution = "'access_points' must contain resolvable addresses; fix the configuration and restart the cont>
Feb 16 16:51:32 asapo-srv09.desy.de systemd[1]: daos_server.service: Main process exited, code=exited, status=1/FAILURE
Feb 16 16:51:32 asapo-srv09.desy.de systemd[1]: daos_server.service: Failed with result 'exit-code'.

I added the "name: daos_server" line, reduced the access_points list to just one node and tried with th ip number, always teh same result.
root@asapo-srv09 ~]# grep -v ^# /etc/daos/daos_server.yml
name: daos_server
access_points: ['131.169.183.155']
provider: ofi+verbs;ofi_rxm
bdev_include: ["01:00.0","02:00.0"]

I use
Name         : daos-server
Version      : 2.0.1
Release      : 2.el8
Architecture : x86_64
Size         : 41 M
Source       : daos-2.0.1-2.el8.src.rpm

on
LSB Version:    :core-4.1-amd64:core-4.1-noarch
Distributor ID:    AlmaLinux
Description:    AlmaLinux release 8.5 (Arctic Sphynx)
Release:    8.5
Codename:    ArcticSphynx

Re: 'access_points' must contain resolvable addresses complaint

Hannappel, Juergen

#1550

Hi,
there is no log file yet, all entries are in the journal.
They look like this:
Feb 16 16:51:32 asapo-srv09.desy.de systemd[1]: Started DAOS Server.
Feb 16 16:51:32 asapo-srv09.desy.de daos_server[267723]: DAOS Server config loaded from /etc/daos/daos_server.yml
Feb 16 16:51:32 asapo-srv09.desy.de daos_server[267723]: no control log file specified; logging to stdout
Feb 16 16:51:32 asapo-srv09.desy.de daos_server[267723]: No DAOS I/O Engines in configuration, DAOS Control Server starting in discovery mode
Feb 16 16:51:32 asapo-srv09.desy.de daos_server[267723]: ERROR: retrieve replicas from config: serverconfig: code = 705 description = "invalid list of access points in configuration"
Feb 16 16:51:32 asapo-srv09.desy.de daos_server[267723]: ERROR: serverconfig: code = 705 resolution = "'access_points' must contain resolvable addresses; fix the configuration and restart the cont>
Feb 16 16:51:32 asapo-srv09.desy.de systemd[1]: daos_server.service: Main process exited, code=exited, status=1/FAILURE
Feb 16 16:51:32 asapo-srv09.desy.de systemd[1]: daos_server.service: Failed with result 'exit-code'.

I added the "name: daos_server" line, reduced the access_points list to just one node and tried with th ip number, always teh same result.
root@asapo-srv09 ~]# grep -v ^# /etc/daos/daos_server.yml
name: daos_server
access_points: ['131.169.183.155']
provider: ofi+verbs;ofi_rxm
bdev_include: ["01:00.0","02:00.0"]

I use
Name         : daos-server
Version      : 2.0.1
Release      : 2.el8
Architecture : x86_64
Size         : 41 M
Source       : daos-2.0.1-2.el8.src.rpm

on
LSB Version:    :core-4.1-amd64:core-4.1-noarch
Distributor ID:    AlmaLinux
Description:    AlmaLinux release 8.5 (Arctic Sphynx)
Release:    8.5
Codename:    ArcticSphynx

Re: 'access_points' must contain resolvable addresses complaint

Nabarro, Tom

#1549

Hello,

I would check if there are any clues from comparing with an example config file (under utils/config/examples dir) and then verify that it works with the IP addresses (maybe just try one to start with and use double quotes like in the examples) just to narrow down the problem.

Regards,

Tom

From: daos@daos.groups.io <daos@daos.groups.io> On Behalf Of Hannappel, Juergen
Sent: Wednesday, February 16, 2022 9:18 AM
To: daos@daos.groups.io
Subject: [daos] 'access_points' must contain resolvable addresses complaint

Hello,
I try to set up daos on a small 4-node cluster, following the recipe on https://docs.daos.io/v2.0/QSG/setup_centos/
When I try to start the server it complains:
Feb 16 09:16:11 asapo-srv09.desy.de daos_server[251023]: ERROR: retrieve replicas from config: serverconfig: code = 705 description = "invalid list of access points in configuration"
Feb 16 09:16:11 asapo-srv09.desy.de daos_server[251023]: ERROR: serverconfig: code = 705 resolution = "'access_points' must contain resolvable addresses; fix the configuration and restart the cont>

Config is:
[root@asapo-srv09 tmp]# grep -v ^# /etc/daos/daos_server.yml
access_points: ['asapo-srv09', 'asapo-srv10', 'asapo-srv11', 'asapo-srv12']
provider: ofi+verbs;ofi_rxm
bdev_include: ["01:00.0","02:00.0"]

The node names are resolvable:
root@asapo-srv09 tmp]# host asapo-srv09
asapo-srv09.desy.de has address 131.169.183.155
[root@asapo-srv09 tmp]# host asapo-srv10
asapo-srv10.desy.de has address 131.169.183.157
... and so on.

The result was the same when I added the domain name to the host names in the config file or used the ip adresses.
Probably a stupid error on my side, any hints?

Thanks in advance

Re: 'access_points' must contain resolvable addresses complaint

Hennecke, Michael

#1548

Hi,

Does it work with a single host in the access_points list?

You should add a line with “name: daos_server” above the access_points.

Anything useful in the daos server logfile?

Best,

Michael

From: daos@daos.groups.io <daos@daos.groups.io> On Behalf Of Hannappel, Juergen
Sent: Wednesday, 16 February 2022 10:18
To: daos@daos.groups.io
Subject: [daos] 'access_points' must contain resolvable addresses complaint

Hello,
I try to set up daos on a small 4-node cluster, following the recipe on https://docs.daos.io/v2.0/QSG/setup_centos/
When I try to start the server it complains:
Feb 16 09:16:11 asapo-srv09.desy.de daos_server[251023]: ERROR: retrieve replicas from config: serverconfig: code = 705 description = "invalid list of access points in configuration"
Feb 16 09:16:11 asapo-srv09.desy.de daos_server[251023]: ERROR: serverconfig: code = 705 resolution = "'access_points' must contain resolvable addresses; fix the configuration and restart the cont>

Config is:
[root@asapo-srv09 tmp]# grep -v ^# /etc/daos/daos_server.yml
access_points: ['asapo-srv09', 'asapo-srv10', 'asapo-srv11', 'asapo-srv12']
provider: ofi+verbs;ofi_rxm
bdev_include: ["01:00.0","02:00.0"]

The node names are resolvable:
root@asapo-srv09 tmp]# host asapo-srv09
asapo-srv09.desy.de has address 131.169.183.155
[root@asapo-srv09 tmp]# host asapo-srv10
asapo-srv10.desy.de has address 131.169.183.157
... and so on.

The result was the same when I added the domain name to the host names in the config file or used the ip adresses.
Probably a stupid error on my side, any hints?

Thanks in advance

Intel Deutschland GmbH
Registered Address: Am Campeon 10, 85579 Neubiberg, Germany
Tel: +49 89 99 8853-0, www.intel.de
Managing Directors: Christin Eisenschmid, Sharon Heck, Tiffany Doon Silva
Chairperson of the Supervisory Board: Nicole Lau
Registered Office: Munich
Commercial Register: Amtsgericht Muenchen HRB 186928

'access_points' must contain resolvable addresses complaint

Hannappel, Juergen

#1547

Hello,
I try to set up daos on a small 4-node cluster, following the recipe on https://docs.daos.io/v2.0/QSG/setup_centos/
When I try to start the server it complains:
Feb 16 09:16:11 asapo-srv09.desy.de daos_server[251023]: ERROR: retrieve replicas from config: serverconfig: code = 705 description = "invalid list of access points in configuration"
Feb 16 09:16:11 asapo-srv09.desy.de daos_server[251023]: ERROR: serverconfig: code = 705 resolution = "'access_points' must contain resolvable addresses; fix the configuration and restart the cont>

Config is:
[root@asapo-srv09 tmp]# grep -v ^# /etc/daos/daos_server.yml
access_points: ['asapo-srv09', 'asapo-srv10', 'asapo-srv11', 'asapo-srv12']
provider: ofi+verbs;ofi_rxm
bdev_include: ["01:00.0","02:00.0"]

The node names are resolvable:
root@asapo-srv09 tmp]# host asapo-srv09
asapo-srv09.desy.de has address 131.169.183.155
[root@asapo-srv09 tmp]# host asapo-srv10
asapo-srv10.desy.de has address 131.169.183.157
... and so on.

The result was the same when I added the domain name to the host names in the config file or used the ip adresses.
Probably a stupid error on my side, any hints?

Thanks in advance

Re: Consistency problem in daos

@stephen.tao

#1546

Are there two situations when trying abort after submission failure：
1. The peer1 node submitted successfully, but did not receive the abort request due to network reasons (Should it go back even if it receive abort request? )
2. The peer2 submission failed and received an abort request, so abort was completed
Is there any inconsistency at this time?

Thansks for your reponding, best regards!

Re: High latency in metada write

shadow_vector@...

#1545

Hi Liang:

Thank you for the concern and interpretation. I get the reason. So I think meta data write in array update interface would take less time than the vos_perf test before due to less data would be written in SCM. Is there something wrong with my understanding? Looking forward to the benchmark result.

Best Regards!

Re: Announcement: DAOS 2.0.1 is generally available

Prantis, Kelsey

#1544

One small error correction due to typo, highlighting that the text below should say *2.0.1* release as the subject line does, not 2.0.2.

Kelsey

From: <daos@daos.groups.io> on behalf of "Prantis, Kelsey" <kelsey.prantis@...>
Reply-To: "daos@daos.groups.io" <daos@daos.groups.io>
Date: Tuesday, February 1, 2022 at 7:14 PM
To: "daos@daos.groups.io" <daos@daos.groups.io>
Subject: [daos] Announcement: DAOS 2.0.1 is generally available

All,

We are pleased to announce that DAOS 2.0.1 release is now generally available. This maintenance release contains the following updates on top of DAOS 2.0.0:

DAOS 2.0.1 includes fixes to the EC, VOS and Object services, as well as improvements to the control system and dfuse. It also includes numerous updates to the test and build infrastructure.
log4j-core has been updated from 2.16.0 to 2.17.1 DAOS-8929.
libfabric has been updated from 1.14.0~rc3-2 to 1.14.0-1. This also fixes the DAOS 2.0.0 known limitation with MOFED > 5.4-1.0.3.0 described in DAOS-9376.
mercury has been updated from 2.1.0~rc4-1 to 2.1.0~rc4-3. This fixes the high CPU utilization issue in DAOS 2.0.0 described in DAOS-9325
spdk has been updated from 21.07-8 to 21.07-11 (minor fixes only).

There are a number of resources available for the release:

RPM Repositories: https://packages.daos.io/v2.0/
Release Notes: https://docs.daos.io/v2.0/release/release_notes/
Admin Guide: https://daos-stack.github.io/admin/hardware/
User Guide: https://docs.daos.io/v2.0/user/workflow/
Architecture Overview: https://docs.daos.io/v2.0/overview/architecture/
Source Code: https://github.com/daos-stack/daos/releases/ (it may take up to a day for the new release tag to appear)

As always, feel free to use this mailing list for any issues you may find with the release, or our JIRA bug tracking system, available at https://daosio.atlassian.net/jira, or on our Slack channel, available at https://daos-stack.slack.com.

Regards,

Kelsey Prantis

Senior Software Engineering Manager

Extreme Storage Architecture and Development Division

Intel

Announcement: DAOS 2.0.1 is generally available

Prantis, Kelsey

#1543

All,

We are pleased to announce that DAOS 2.0.2 release is now generally available. This maintenance release contains the following updates on top of DAOS 2.0.0:

DAOS 2.0.1 includes fixes to the EC, VOS and Object services, as well as improvements to the control system and dfuse. It also includes numerous updates to the test and build infrastructure.
log4j-core has been updated from 2.16.0 to 2.17.1 DAOS-8929.
libfabric has been updated from 1.14.0~rc3-2 to 1.14.0-1. This also fixes the DAOS 2.0.0 known limitation with MOFED > 5.4-1.0.3.0 described in DAOS-9376.
mercury has been updated from 2.1.0~rc4-1 to 2.1.0~rc4-3. This fixes the high CPU utilization issue in DAOS 2.0.0 described in DAOS-9325
spdk has been updated from 21.07-8 to 21.07-11 (minor fixes only).

There are a number of resources available for the release:

RPM Repositories: https://packages.daos.io/v2.0/
Release Notes: https://docs.daos.io/v2.0/release/release_notes/
Admin Guide: https://daos-stack.github.io/admin/hardware/
User Guide: https://docs.daos.io/v2.0/user/workflow/
Architecture Overview: https://docs.daos.io/v2.0/overview/architecture/
Source Code: https://github.com/daos-stack/daos/releases/ (it may take up to a day for the new release tag to appear)

As always, feel free to use this mailing list for any issues you may find with the release, or our JIRA bug tracking system, available at https://daosio.atlassian.net/jira, or on our Slack channel, available at https://daos-stack.slack.com.

Regards,

Kelsey Prantis

Senior Software Engineering Manager

Extreme Storage Architecture and Development Division

Intel

Re: Consistency problem in daos

Lombardi, Johann

#1542

Hi there,

If abort is called either by the application directly (daos_tx_abort()) or by DAOS internally (something went wrong during commit), then all changes associated with the transaction are discarded and consistency is preserved. None of the uncommitted modifications are visible to external readers.

Cheers,

Johann

From: <daos@daos.groups.io> on behalf of "stephen.tao@..." <stephen.tao@...>
Reply-To: "daos@daos.groups.io" <daos@daos.groups.io>
Date: Saturday 29 January 2022 at 10:47
To: "daos@daos.groups.io" <daos@daos.groups.io>
Subject: [daos] Consistency problem in daos

1. Will abort be triggered if a transaction fails in the commit phase in Daos? (I read the code myself and found that this happens when the submission fails. At this time, some nodes may submit successfully)
2. If abort is triggered, what is the meaning? Will strong consistency be lost?

---------------------------------------------------------------------
Intel Corporation SAS (French simplified joint stock company)
Registered headquarters: "Les Montalets"- 2, rue de Paris,
92196 Meudon Cedex, France
Registration Number: 302 456 199 R.C.S. NANTERRE
Capital: 4,572,000 Euros

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

Consistency problem in daos

@stephen.tao

#1541

1. Will abort be triggered if a transaction fails in the commit phase in Daos? (I read the code myself and found that this happens when the submission fails. At this time, some nodes may submit successfully)
2. If abort is triggered, what is the meaning? Will strong consistency be lost?