Date   

Re: Jenkins test

dongfeier
 
Edited

Hello,
     
useradd: UID 0 is not unique
The command '/bin/sh -c useradd --no-log-init --uid $UID --user-group --create-home --shell /bin/bash             --home /home/daos daos_server' returned a non-zero code: 4

Jenkins not configured to notify users of failed builds.
Scripts not permitted to use staticMethod org.codehaus.groovy.runtime.DefaultGroovyMethods getAt java.lang.Object java.lang.String. Administrators can decide whether to approve or reject this signature.
Error when executing unsuccessful post condition:
org.jenkinsci.plugins.scriptsecurity.sandbox.RejectedAccessException: Scripts not permitted to use staticMethod org.codehaus.groovy.runtime.DefaultGroovyMethods getAt java.lang.Object java.lang.String
	at org.jenkinsci.plugins.scriptsecurity.sandbox.whitelists.StaticWhitelist.rejectStaticMethod(StaticWhitelist.java:279)
	at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onGetArray(SandboxInterceptor.java:476)
	at org.kohsuke.groovy.sandbox.impl.Checker$11.call(Checker.java:484)
	at org.kohsuke.groovy.sandbox.impl.Checker.checkedGetArray(Checker.java:489)
	at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.getArray(SandboxInvoker.java:45)
	at com.cloudbees.groovy.cps.impl.ArrayAccessBlock.rawGet(ArrayAccessBlock.java:21)
	at notifyBrokenBranch.call(notifyBrokenBranch.groovy:37)
	at WorkflowScript.run(WorkflowScript:1049)
	at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.delegateAndExecute(ModelInterpreter.groovy:137)
	at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.runPostConditions(ModelInterpreter.groovy:761)
	at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.catchRequiredContextForNode(ModelInterpreter.groovy:395)
	at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.catchRequiredContextForNode(ModelInterpreter.groovy:393)
	at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.runPostConditions(ModelInterpreter.groovy:760)
	at com.cloudbees.groovy.cps.CpsDefaultGroovyMethods.each(CpsDefaultGroovyMethods:2030)
	at com.cloudbees.groovy.cps.CpsDefaultGroovyMethods.each(CpsDefaultGroovyMethods:2015)
	at com.cloudbees.groovy.cps.CpsDefaultGroovyMethods.each(CpsDefaultGroovyMethods:2056)
	at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.runPostConditions(ModelInterpreter.groovy:750)
	at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.runPostConditions(ModelInterpreter.groovy)
	at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.executePostBuild(ModelInterpreter.groovy:728)
	at ___cps.transform___(Native Method)
	at com.cloudbees.groovy.cps.impl.PropertyishBlock$ContinuationImpl.get(PropertyishBlock.java:74)
	at com.cloudbees.groovy.cps.LValueBlock$GetAdapter.receive(LValueBlock.java:30)
	at com.cloudbees.groovy.cps.impl.PropertyishBlock$ContinuationImpl.fixName(PropertyishBlock.java:66)
	at sun.reflect.GeneratedMethodAccessor430.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
	at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21)
	at com.cloudbees.groovy.cps.Next.step(Next.java:83)
	at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174)
	at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163)
	at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:129)
	at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:268)
	at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163)
	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18)
	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:51)
	at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:185)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:402)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$400(CpsThreadGroup.java:96)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:314)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:278)
	at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:67)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:139)
	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
	at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
I have found the cause of the problem because the uid is repeated when adding users. Thank you very much


DAOS Community Update / Mar'22

Lombardi, Johann
 

Hi there,

 

Please find below the DAOS community newsletter for March 2022.

 

Past Events (February)

 

Upcoming Events

 

Release

  • Current stable release is 2.0.1 and 2.0.2 is under development.
  • Master is the development branch for the future 2.2 release.
  • Major recent 2.0 changes to be available in 2.0.2:
    • Build go code shipped in RPM with go >= 1.17
    • Fix a bug with key enumeration and multiple shards
    • Many documentation fixes (e.g. fix broken links)
    • Use larger ABT stack for NVMe poll ULT
    • Reject integer keys that are smaller than 8 bytes
    • Update the C-based raft implementation to pick a membership fix
    • Disable dfuse-level readahead when FUSE is configured in writeback mode and already drive readahead from the kernel.
  • Major recent master changes:
    • Fix a problem between mercury and libfabric causing some bulk transfers to be stuck indefinitely
    • Reduce CPU load caused by the engine when DAOS system is idle
    • Add support for owner/group to libdfs
    • Automatically calculate the required number of huge pages
    • Several VMD and SSD hotplug fixes
    • Document the need to disable shared received context for Intel E810-C cards
    • Rewrite of the netdetect module
    • Add UCX support to the manual (non RPM) build infrastructure
  • What is coming:
    • 2.0.2 validation
    • 2.2.0 feature freeze

 

R&D

  • Major features under development:
  • Pathfinding:
    • MariaDB DAOS engine with predicate pushdown to the DAOS storage nodes
    • Leveraging the Intel Data Streaming Accelerator (DSA) to accelerate DAOS
      • Prototype leveraging DSA for VOS aggregation delivered
    • S3 support via a DAOS backend to Rados Gateway (RGW)
    • GPU data path optimizations

 

News

See https://events.linuxfoundation.org/sodacode/ for more information.

 

---------------------------------------------------------------------
Intel Corporation SAS (French simplified joint stock company)
Registered headquarters: "Les Montalets"- 2, rue de Paris,
92196 Meudon Cedex, France
Registration Number:  302 456 199 R.C.S. NANTERRE
Capital: 4,572,000 Euros

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.


Re: Jenkins test

Murrell, Brian
 

On Wed, 2022-03-02 at 01:38 -0800, dongfeier wrote:
Hello,
After the Jenkins environment configuration is completed, the test is
carried out, and the following exceptions are found
Some function needs to be moved to the trusted library. There is not
enough information in your screenshot to know which function though.
For this reason copy and paste of text is preferred to screenshots as
it typically carries much more information, particularly in the case of
java stacktraces.

Cheers,
b.


Re: Jenkins test

dongfeier
 

Hello,
     After the Jenkins environment configuration is completed, the test is carried out, and the following exceptions are found
     


Please let me know if you have any suggestions.
Thanks in advance


Re: Jenkins test

Murrell, Brian
 

On Mon, 2022-02-28 at 17:45 -0800, 15735154041@... wrote:
Hello,
Hi,

Jenkins test uses the pipeline shared library. After the
configuration is completed, there is the following exception "no such
DSL method 'commitpragmatrusted' found among steps". Please let me
know if you have any suggestions.
That is implemented in:
https://github.com/daos-stack/trusted-pipeline-lib/blob/master/vars/commitPragmaTrusted.groovy

which is supplied by the:

https://github.com/daos-stack/trusted-pipeline-lib

hared library. You need to apply that library as a trusted library to
your Jenkins configuration as instructed at:

https://www.jenkins.io/doc/book/pipeline/shared-libraries/#global-shared-libraries

Cheers,
b.


Re: Jenkins test

dongfeier
 

Hello,
     Jenkins test uses the pipeline shared library. After the configuration is completed, there is the following exception "no such DSL method 'commitpragmatrusted' found among steps". Please let me know if you have any suggestions.
Thanks in advance


Re: Jenkins test

Lombardi, Johann
 

Hi there,

Please find below the plugin usage page from Jenkins. HTH.

Cheers,
Johann

Warning: Scripted pipeline jobs are not detected by this plugin. Support for declarative pipeline jobs is experimental. Number of Jobs: 46 


Jobs by Plugin

Plugin Name

Version

Number of Jobs

Jobs

Has Dependants?

AnsiColor

1.0.0

0

false

Ant Plugin

1.11

0

true

Build Blocker Plugin

1.7.7

12

false

Build Failure Analyzer

2.0.1

968

true

Build Name and Description Setter

2.2.0

0

false

Build Pipeline Plugin

1.5.8

0

false

Build Timeout

1.20

43

false

Checks API plugin

1.7.2

0

true

Code Coverage API Plugin

1.4.1

0

true

Conditional BuildStep

1.4.1

0

true

Config File Provider Plugin

3.8.1

0

true

Confluence Publisher

2.0.6

0

false

Copy Artifact Plugin

1.46.1

0

true

Credentials Binding Plugin

1.27

1

true

Docker Pipeline

1.26

0

false

Email Extension Plugin

2.83

0

true

Email Extension Template Plugin

1.2

0

false

Environment Injector Plugin

2.4.0

0

true

File Operations Plugin

1.11

0

false

Folders Plugin

6.16

0

true

Forensics API Plugin

1.3.0

0

true

Git Forensics Plugin

1.2.0

0

false

GitHub plugin

1.34.1

0

true

GitLab Plugin

1.5.20

16

false

Gradle Plugin

1.37.1

0

true

Groovy

2.4

0

false

HTML Publisher plugin

1.25

0

true

HTTP Request Plugin

1.11

0

false

Ivy Plugin

2.1

0

true

JUnit Plugin

1.53

0

true

Javadoc Plugin

1.6

0

true

Jenkins Artifactory Plugin

3.14.1

0

false

Jenkins BuildResultTrigger Plug-in

0.17

0

false

Jenkins CVS Plug-in

2.19

0

true

Jenkins Clover plugin

4.12.1

0

false

Jenkins Cobertura Plugin

1.16

0

false

Jenkins Cppcheck Plug-in

1.25

0

false

Jenkins Doxygen Plug-in

0.18

0

false

Jenkins Git plugin

4.8.2

2

true

Jenkins Jira plugin

3.5

7

true

Jenkins Job Configuration History Plugin

2.28.1

3

true

Jenkins Libvirt Agents plugin

1.9.2

12

false

Jenkins Mailer Plugin

1.34

0

true

Jenkins Mercurial plugin

2.15

0

false

Jenkins Multijob plugin

1.36

0

true

Jenkins Parameterized Trigger plugin

2.41

0

true

Jenkins PostBuildScript Plugin

3.0.0

0

false

Jenkins SSH plugin

2.6.1

0

false

Jenkins Subversion Plug-in

2.14.4

0

true

Jenkins Throttle Concurrent Builds Plug-in

2.4

12

false

Jenkins Valgrind Plug-in

0.28

0

false

Jenkins Workspace Cleanup Plugin

0.39

0

false

Job and Node ownership plugin

0.13.0

689

false

Lockable Resources plugin

2.11

0

true

Managed Scripts

1.5.4

0

false

Mask Passwords Plugin

3.0

0

false

Matrix Authorization Strategy Plugin

2.6.8

8

true

Maven Integration plugin

3.12

0

true

Pipeline GitHub Notify Step Plugin

1.0.5

0

false

Pipeline Utility Steps

2.10.0

0

false

Pipeline: Basic Steps

2.24

0

true

Pipeline: Build Step

2.15

0

true

Pipeline: Declarative

1.9.1

431

true

Pipeline: GitHub

2.7

0

false

Pipeline: Groovy

2.93

0

true

Pipeline: Input Step

2.12

0

true

Pipeline: Job

2.41

410

true

Pipeline: Milestone Step

1.3.2

0

true

Pipeline: Multibranch

2.26

923

true

Pipeline: Nodes and Processes

2.39

0

true

Pipeline: SCM Step

2.13

0

true

Pipeline: Shared Groovy Libraries

2.21

0

true

Pipeline: Stage Step

2.5

0

true

Plot plugin

2.1.9

0

false

Post build task

1.9

0

false

Publish Over SSH

1.22

0

false

Rebuilder

1.32

12

true

SSH Agent Plugin

1.23

2

false

SSH Pipeline Steps

2.0.0

0

false

Timestamper

1.13

45

false

Token Macro Plugin

266.v44a80cf277fd

0

true

Warnings Next Generation Plugin

9.5.0

0

false

xUnit plugin

3.0.2

0

false

 


Other Plugins

Plugin Name

Version

Has Dependants?

Active Choices Plug-in

2.5.6

false

Analysis Model API Plugin

10.3.0

true

Audit Trail

3.10

false

Authentication Tokens API Plugin

1.4

true

Autofavorite for Blue Ocean

1.2.4

true

Basic Branch Build Strategies Plugin

1.3.2

false

Bitbucket Branch Source Plugin

2.9.11

true

Bitbucket Pipeline for Blue Ocean

1.24.8

true

Blue Ocean

1.24.8

false

Blue Ocean Core JS

1.24.8

true

Blue Ocean Pipeline Editor

1.24.8

true

Bootstrap 4 API Plugin

4.6.0-3

true

Bootstrap 5 API Plugin

5.1.0-3

true

Branch API Plugin

2.6.5

true

Build Blocker Plugin

1.7.7

false

Build Failure Analyzer

2.0.1

true

Build History Metrics plugin

93.v0e17adc6ccea

false

Build Monitor View

1.12+build.201809061734

false

Caffeine API Plugin

2.9.2-29.v717aac953ff3

true

Command Agent Launcher Plugin

1.6

true

Common API for Blue Ocean

1.24.8

true

Config API for Blue Ocean

1.24.8

true

Config AutoRefresh Plugin

1.0

false

Credentials Plugin

2.5

true

DTKit 2 API.

3.0.0

true

Dashboard for Blue Ocean

1.24.8

true

DataTables.net API Plugin

1.10.25-3

true

Disable GitHub Multibranch Status Plugin

1.2

false

Display URL API

2.3.5

true

Display URL for Blue Ocean

2.4.1

true

Docker Commons Plugin

1.17

true

Durable Task Plugin

1.39

true

ECharts API Plugin

5.1.2-11

true

EZ Templates

1.3.4

false

EnvInject API Plugin

1.7

true

Events API for Blue Ocean

1.24.8

true

External Monitor Job Type Plugin

1.7

true

Favorite

2.3.3

true

Font Awesome API Plugin

5.15.4-1

true

Git Parameter Plug-In

0.9.13

false

Git Pipeline for Blue Ocean

1.24.8

true

GitHub API Plugin

1.123

true

GitHub Branch Source Plugin

2.11.2

true

GitHub Pipeline for Blue Ocean

1.24.8

true

Green Balls

1.15.1

false

Handy Uri Templates 2.x API Plugin

2.1.8-1.0

true

Hudson Personal View

1.8

false

Hudson global-build-stats plugin

1.5

true

Infrastructure plugin for Publish Over X

0.22

true

JAXB plugin

2.3.0.1

true

JIRA Integration for Blue Ocean

1.24.8

true

JQuery3 API Plugin

3.6.0-2

true

JWT for Blue Ocean

1.24.8

true

Jackson 2 API Plugin

2.12.4

true

Java JSON Web Token (JJWT) Plugin

0.11.2-9.c8b45b8bb173

true

JavaScript GUI Lib: ACE Editor bundle plugin

1.1

true

JavaScript GUI Lib: Handlebars bundle plugin

3.0.8

true

JavaScript GUI Lib: Moment.js bundle plugin

1.1.1

true

JavaScript GUI Lib: jQuery bundles (jQuery and jQuery UI) plugin

1.2.1

true

Jenkins Apache HttpComponents Client 4.x API Plugin

4.5.13-1.0

true

Jenkins Design Language

1.24.8

true

Jenkins Extended Read Permission Plugin

3.2

false

Jenkins GIT server Plugin

1.10

true

Jenkins Git client plugin

3.9.0

true

Jenkins JSch dependency plugin

0.1.55.2

true

Jenkins Job Configuration History Plugin

2.28.1

true

Jenkins Libvirt Agents plugin

1.9.2

false

Jenkins Pipeline Remote Loader Plugin

1.5

false

Jenkins Platform Labeler Plugin

987.v674996136faf

false

Jenkins Priority Sorter Plugin

4.0.0

false

Jenkins Pub-Sub "light" Bus

1.16

true

Jenkins Slave SetupPlugin

1.10

false

JenkinsLint Plugin

0.14.0

false

Job Import Plugin

3.4

false

LDAP Plugin

2.7

true

Lenient shutdown plugin

1.1.1

false

Mail Watcher Plugin

1.16

false

MapDB API Plugin

1.0.9.0

true

Matrix Authorization Strategy Plugin

2.6.8

true

Matrix Project Plugin

1.19

true

Matrix Reloaded Plugin

1.1.3

false

Metrics Plugin

4.0.2.8

true

Monitoring

1.88.0

false

Next Build Number Plugin

1.6

false

OWASP Markup Formatter Plugin

2.1

true

OkHttp Plugin

3.14.9

true

Oracle Java SE Development Kit Installer Plugin

1.5

true

PAM Authentication plugin

1.6

true

Persistent Parameter Plugin

1.3

false

Personalization for Blue Ocean

1.24.8

true

Pipeline

2.6

true

Pipeline Graph Analysis Plugin

1.11

true

Pipeline SCM API for Blue Ocean

1.24.8

true

Pipeline implementation for Blue Ocean

1.24.8

true

Pipeline: API

2.46

true

Pipeline: Declarative Extension Points API

1.9.1

true

Pipeline: GitHub Groovy Libraries

1.0

false

Pipeline: Job

2.41

true

Pipeline: Model API

1.9.1

true

Pipeline: REST API Plugin

2.19

true

Pipeline: Stage Tags Metadata

1.9.1

true

Pipeline: Stage View Plugin

2.19

true

Pipeline: Step API

2.24

true

Pipeline: Supporting APIs

3.8

true

Plain Credentials Plugin

1.7

true

Plugin Usage - Plugin

1.2

false

Plugin Utilities API Plugin

2.4.0

true

Popper.js 2 API Plugin

2.9.3-1

true

Popper.js API Plugin

1.16.1-2

true

Prometheus metrics plugin

2.0.10

false

REST API for Blue Ocean

1.24.8

true

REST Implementation for Blue Ocean

1.24.8

true

Rebuilder

1.32

true

Resource Disposer Plugin

0.16

true

Role-based Authorization Strategy

3.2.0

true

Run Condition Plugin

1.5

true

SCM API Plugin

2.6.5

true

SSH Build Agents plugin

1.33.0

false

SSH Credentials Plugin

1.19

true

SSH server

3.1.0

true

Script Security Plugin

1.78

true

Server Sent Events (SSE) Gateway Plugin

1.24

true

Sidebar Link

1.12.0

false

Simple Queue Plugin

1.4.1

false

Snakeyaml API Plugin

1.29.1

true

Strict Crumb Issuer Plugin

2.1.0

false

Structs Plugin

1.23

true

Test Results Analyzer Plugin

0.3.5

false

ThinBackup

1.10

false

Trilead API Plugin

1.0.13

true

Variant Plugin

1.4

true

View Job Filters

2.3

true

WMI Windows Agents Plugin

1.8

true

Web for Blue Ocean

1.24.8

true

bouncycastle API Plugin

2.24

true

build-metrics

1.3

false

built-on-column

1.1

true

excludeMatrixParent

1.1

false

i18n for Blue Ocean

1.24.8

true

jQuery plugin

1.12.4-1

true

 

 

From: <daos@daos.groups.io> on behalf of "15735154041@..." <15735154041@...>
Reply-To: "daos@daos.groups.io" <daos@daos.groups.io>
Date: Monday 28 February 2022 at 06:32
To: "daos@daos.groups.io" <daos@daos.groups.io>
Subject: [daos] Jenkins test

 

Hello everyone:
    Recently, when I deployed the Jenkins test environment, a similar problem occurred after the task was executed: "no such DSL method 'valgrindreportpublish' found among steps". I look forward to providing a list of Jenkins test dependent plug-ins.
Best regards!

---------------------------------------------------------------------
Intel Corporation SAS (French simplified joint stock company)
Registered headquarters: "Les Montalets"- 2, rue de Paris,
92196 Meudon Cedex, France
Registration Number:  302 456 199 R.C.S. NANTERRE
Capital: 4,572,000 Euros

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.


Jenkins test

dongfeier
 

Hello everyone:
    Recently, when I deployed the Jenkins test environment, a similar problem occurred after the task was executed: "no such DSL method 'valgrindreportpublish' found among steps". I look forward to providing a list of Jenkins test dependent plug-ins.
Best regards!


Re: 'access_points' must contain resolvable addresses complaint

Hannappel, Juergen
 

The config file was created by modifying the file that was installed from the RPM, what I showd in the mail were just the lines which are not commented out.
I commented out the bdev_include, but that did not help.

See here the comple diff to the version in utils/config  (why is that one different from the one installed initailly in /etc/daos ?):
[root@asapo-srv09 ~]# diff /usr/local/src/daos/utils/config/daos_server.yml /etc/daos/daos_server.yml
15c15
< #name: daos_server
---
> name: daos_server
27c27
< #access_points: ['hostname1']
---
> access_points: ['131.169.183.155']
75,76c75,76
< ##  ofi+verbs for Infiniband/RoCE and
< ##  ofi+tcp for non-RDMA-capable Ethernet.
---
> ##  ofi+verbs;ofi_rxm for Infiniband/RoCE and
> ##  ofi+sockets for non-RDMA-capable Ethernet.
78c78
< #provider: ofi+verbs
---
> provider: ofi+verbs;ofi_rxm
94,99d93
< ## CART: Disable SRX
< ## parameters shared with client.
< #
< #disable_srx: false
< #
< #
107c101
< #bdev_include: ["0000:81:00.1","0000:81:00.2","0000:81:00.3"]
---
> #bdev_include: ["01:00.0","02:00.0"]
176c170
< ## Number of hugepages to allocate for DMA buffer memory
---
> ## Number of hugepages to allocate for use by NVMe SSDs
179,189c173,175
< ## through SPDK. Note that each target requires 1 GB of hugepage space, so
< ## this value needs to represent the total amount of hugespace required for
< ## all targets across all engines on host, divided by the system hugepage size.
< ## If not set here, an appropriate value will be automatically calculated and
< ## default system hugepage size will be used.
< #
< ## Example: (2 engines * (8 targets/engine * 1GB)) / 2MB hugepage size = 16834
< #
< ## Hugepages are mandatory with NVME SSDs configured and optional without.
< ## To disabled the use of hugepages when no NVMe SSDs are configured, set
< ## nr_hugepages to -1.
---
> ## through SPDK. This indicates the number to be used for each spawned
> ## I/O Engines, so the total will be this number * number of I/O Engines.
> ## Default system hugepage size will be used.
191c177,178
< ##nr_hugepages: -1
---
> ## default: 4096
> #nr_hugepages: 4096
353,354c340
< #    bdev_list: ["0000:81:00.0", "0000:82:00.0"]  # generate regular nvme.conf
< #
---
> #    bdev_list: ["0000:81:00.0"]  # generate regular nvme.conf
359,365d344
< #
< #    # Optional override, will be automatically generated based on NUMA affinity.
< #    # Filter hot-pluggable devices by PCI bus-ID by specifying a hexadecimal
< #    # range. Hotplug events relating to devices with PCI bus-IDs outside this range
< #    # will not be processed by this engine. Empty or unset range signifies allow all.
< #    bdev_busid_range: 0x80-0x8f
< #    #bdev_busid_range: 128-143
501,512c480,481
< #    #class: nvme
< #    #bdev_list: ["0000:5d:05.5"]
< #
< #    #class: nvme
< #    #bdev_list: ["0000:da:00.0", "0000:db:00.0"]  # generate regular nvme.conf
< #
< #    # Optional override, will be automatically generated based on NUMA affinity.
< #    # Filter hot-pluggable devices by PCI bus-ID by specifying a hexadecimal
< #    # range. Hotplug events relating to devices with PCI bus-IDs outside this range
< #    # will not be processed by this engine. Empty or unset range signifies allow all.
< #    #bdev_busid_range: 0xd0-0xdf
< #    #bdev_busid_range: 208-223
---
> #    class: nvme
> #    bdev_list: ["0000:82:00.0","0000:5d:05.5"]


Re: 'access_points' must contain resolvable addresses complaint

Nabarro, Tom
 

Hello,

 

Could you please try to use one of the examples in utils/config/examples and look at the reference documentation in utils/config/daos_server.yml. The config file you are using below is not valid. Specifically bdev_include is not a global parameter.

 

Regards,

Tom

 

From: daos@daos.groups.io <daos@daos.groups.io> On Behalf Of Hannappel, Juergen
Sent: Wednesday, February 16, 2022 3:56 PM
To: daos@daos.groups.io
Subject: Re: [daos] 'access_points' must contain resolvable addresses complaint

 

Hi,
there is no log file yet, all entries are in the journal.
They look like this:
Feb 16 16:51:32 asapo-srv09.desy.de systemd[1]: Started DAOS Server.
Feb 16 16:51:32 asapo-srv09.desy.de daos_server[267723]: DAOS Server config loaded from /etc/daos/daos_server.yml
Feb 16 16:51:32 asapo-srv09.desy.de daos_server[267723]: no control log file specified; logging to stdout
Feb 16 16:51:32 asapo-srv09.desy.de daos_server[267723]: No DAOS I/O Engines in configuration, DAOS Control Server starting in discovery mode
Feb 16 16:51:32 asapo-srv09.desy.de daos_server[267723]: ERROR: retrieve replicas from config: serverconfig: code = 705 description = "invalid list of access points in configuration"
Feb 16 16:51:32 asapo-srv09.desy.de daos_server[267723]: ERROR: serverconfig: code = 705 resolution = "'access_points' must contain resolvable addresses; fix the configuration and restart the cont>
Feb 16 16:51:32 asapo-srv09.desy.de systemd[1]: daos_server.service: Main process exited, code=exited, status=1/FAILURE
Feb 16 16:51:32 asapo-srv09.desy.de systemd[1]: daos_server.service: Failed with result 'exit-code'.

I added the "name: daos_server" line, reduced the access_points list to just one node and tried with th ip number, always teh same result.
root@asapo-srv09 ~]# grep -v ^# /etc/daos/daos_server.yml
name: daos_server
access_points: ['131.169.183.155']
provider: ofi+verbs;ofi_rxm
bdev_include: ["01:00.0","02:00.0"]


I use
Name         : daos-server
Version      : 2.0.1
Release      : 2.el8
Architecture : x86_64
Size         : 41 M
Source       : daos-2.0.1-2.el8.src.rpm

on
LSB Version:    :core-4.1-amd64:core-4.1-noarch
Distributor ID:    AlmaLinux
Description:    AlmaLinux release 8.5 (Arctic Sphynx)
Release:    8.5
Codename:    ArcticSphynx


Re: 'access_points' must contain resolvable addresses complaint

Hannappel, Juergen
 

Hi,
there is no log file yet, all entries are in the journal.
They look like this:
Feb 16 16:51:32 asapo-srv09.desy.de systemd[1]: Started DAOS Server.
Feb 16 16:51:32 asapo-srv09.desy.de daos_server[267723]: DAOS Server config loaded from /etc/daos/daos_server.yml
Feb 16 16:51:32 asapo-srv09.desy.de daos_server[267723]: no control log file specified; logging to stdout
Feb 16 16:51:32 asapo-srv09.desy.de daos_server[267723]: No DAOS I/O Engines in configuration, DAOS Control Server starting in discovery mode
Feb 16 16:51:32 asapo-srv09.desy.de daos_server[267723]: ERROR: retrieve replicas from config: serverconfig: code = 705 description = "invalid list of access points in configuration"
Feb 16 16:51:32 asapo-srv09.desy.de daos_server[267723]: ERROR: serverconfig: code = 705 resolution = "'access_points' must contain resolvable addresses; fix the configuration and restart the cont>
Feb 16 16:51:32 asapo-srv09.desy.de systemd[1]: daos_server.service: Main process exited, code=exited, status=1/FAILURE
Feb 16 16:51:32 asapo-srv09.desy.de systemd[1]: daos_server.service: Failed with result 'exit-code'.

I added the "name: daos_server" line, reduced the access_points list to just one node and tried with th ip number, always teh same result.
root@asapo-srv09 ~]# grep -v ^# /etc/daos/daos_server.yml
name: daos_server
access_points: ['131.169.183.155']
provider: ofi+verbs;ofi_rxm
bdev_include: ["01:00.0","02:00.0"]


I use
Name         : daos-server
Version      : 2.0.1
Release      : 2.el8
Architecture : x86_64
Size         : 41 M
Source       : daos-2.0.1-2.el8.src.rpm

on
LSB Version:    :core-4.1-amd64:core-4.1-noarch
Distributor ID:    AlmaLinux
Description:    AlmaLinux release 8.5 (Arctic Sphynx)
Release:    8.5
Codename:    ArcticSphynx


Re: 'access_points' must contain resolvable addresses complaint

Nabarro, Tom
 

Hello,

 

I would check if there are any clues from comparing with an example config file (under utils/config/examples dir) and then verify that it works with the IP addresses (maybe just try one to start with and use double quotes like in the examples) just to narrow down the problem.

 

Regards,

Tom

 

From: daos@daos.groups.io <daos@daos.groups.io> On Behalf Of Hannappel, Juergen
Sent: Wednesday, February 16, 2022 9:18 AM
To: daos@daos.groups.io
Subject: [daos] 'access_points' must contain resolvable addresses complaint

 

Hello,
I try to set up daos on a small 4-node cluster, following the recipe on https://docs.daos.io/v2.0/QSG/setup_centos/
When I try to start the server it complains:
Feb 16 09:16:11 asapo-srv09.desy.de daos_server[251023]: ERROR: retrieve replicas from config: serverconfig: code = 705 description = "invalid list of access points in configuration"
Feb 16 09:16:11 asapo-srv09.desy.de daos_server[251023]: ERROR: serverconfig: code = 705 resolution = "'access_points' must contain resolvable addresses; fix the configuration and restart the cont>

Config is:
[root@asapo-srv09 tmp]# grep -v ^# /etc/daos/daos_server.yml
access_points: ['asapo-srv09', 'asapo-srv10', 'asapo-srv11', 'asapo-srv12']
provider: ofi+verbs;ofi_rxm
bdev_include: ["01:00.0","02:00.0"]

The node names are resolvable:
root@asapo-srv09 tmp]# host asapo-srv09
asapo-srv09.desy.de has address 131.169.183.155
[root@asapo-srv09 tmp]# host asapo-srv10
asapo-srv10.desy.de has address 131.169.183.157
... and so on.

The result was the same when I added the domain name to the host names in the config file or used the ip adresses.
Probably a stupid error on my side, any hints?

Thanks in advance


Re: 'access_points' must contain resolvable addresses complaint

Hennecke, Michael
 

Hi,

 

Does it work with a single host in the access_points list? 

You should add a line with “name: daos_server” above the access_points.

Anything useful in the daos server logfile?

 

Best,

Michael

 

From: daos@daos.groups.io <daos@daos.groups.io> On Behalf Of Hannappel, Juergen
Sent: Wednesday, 16 February 2022 10:18
To: daos@daos.groups.io
Subject: [daos] 'access_points' must contain resolvable addresses complaint

 

Hello,
I try to set up daos on a small 4-node cluster, following the recipe on https://docs.daos.io/v2.0/QSG/setup_centos/
When I try to start the server it complains:
Feb 16 09:16:11 asapo-srv09.desy.de daos_server[251023]: ERROR: retrieve replicas from config: serverconfig: code = 705 description = "invalid list of access points in configuration"
Feb 16 09:16:11 asapo-srv09.desy.de daos_server[251023]: ERROR: serverconfig: code = 705 resolution = "'access_points' must contain resolvable addresses; fix the configuration and restart the cont>

Config is:
[root@asapo-srv09 tmp]# grep -v ^# /etc/daos/daos_server.yml
access_points: ['asapo-srv09', 'asapo-srv10', 'asapo-srv11', 'asapo-srv12']
provider: ofi+verbs;ofi_rxm
bdev_include: ["01:00.0","02:00.0"]

The node names are resolvable:
root@asapo-srv09 tmp]# host asapo-srv09
asapo-srv09.desy.de has address 131.169.183.155
[root@asapo-srv09 tmp]# host asapo-srv10
asapo-srv10.desy.de has address 131.169.183.157
... and so on.

The result was the same when I added the domain name to the host names in the config file or used the ip adresses.
Probably a stupid error on my side, any hints?

Thanks in advance

Intel Deutschland GmbH
Registered Address: Am Campeon 10, 85579 Neubiberg, Germany
Tel: +49 89 99 8853-0, www.intel.de
Managing Directors: Christin Eisenschmid, Sharon Heck, Tiffany Doon Silva  
Chairperson of the Supervisory Board: Nicole Lau
Registered Office: Munich
Commercial Register: Amtsgericht Muenchen HRB 186928


'access_points' must contain resolvable addresses complaint

Hannappel, Juergen
 

Hello,
I try to set up daos on a small 4-node cluster, following the recipe on https://docs.daos.io/v2.0/QSG/setup_centos/
When I try to start the server it complains:
Feb 16 09:16:11 asapo-srv09.desy.de daos_server[251023]: ERROR: retrieve replicas from config: serverconfig: code = 705 description = "invalid list of access points in configuration"
Feb 16 09:16:11 asapo-srv09.desy.de daos_server[251023]: ERROR: serverconfig: code = 705 resolution = "'access_points' must contain resolvable addresses; fix the configuration and restart the cont>

Config is:
[root@asapo-srv09 tmp]# grep -v ^# /etc/daos/daos_server.yml
access_points: ['asapo-srv09', 'asapo-srv10', 'asapo-srv11', 'asapo-srv12']
provider: ofi+verbs;ofi_rxm
bdev_include: ["01:00.0","02:00.0"]

The node names are resolvable:
root@asapo-srv09 tmp]# host asapo-srv09
asapo-srv09.desy.de has address 131.169.183.155
[root@asapo-srv09 tmp]# host asapo-srv10
asapo-srv10.desy.de has address 131.169.183.157
... and so on.

The result was the same when I added the domain name to the host names in the config file or used the ip adresses.
Probably a stupid error on my side, any hints?

Thanks in advance


Re: Consistency problem in daos

@stephen.tao
 

Are there two situations when trying abort after submission failure:
1. The peer1 node submitted successfully, but did not receive the abort request due to network reasons (Should it go back even if it receive abort request? )
2. The peer2 submission failed and received an abort request, so abort was completed
Is there any inconsistency at this time? 

Thansks for your reponding, best regards!


Re: High latency in metada write

shadow_vector@...
 

Hi Liang:

   Thank you for the concern and interpretation. I get the reason. So I think meta data write in array update interface would take less time than the vos_perf test before due to less data would be written in SCM. Is there something wrong with my understanding? Looking forward to the benchmark result.

Best Regards!


Re: Announcement: DAOS 2.0.1 is generally available

Prantis, Kelsey
 

One small error correction due to typo, highlighting that the text below should say *2.0.1* release as the subject line does, not 2.0.2.

 

Kelsey

 

From: <daos@daos.groups.io> on behalf of "Prantis, Kelsey" <kelsey.prantis@...>
Reply-To: "daos@daos.groups.io" <daos@daos.groups.io>
Date: Tuesday, February 1, 2022 at 7:14 PM
To: "daos@daos.groups.io" <daos@daos.groups.io>
Subject: [daos] Announcement: DAOS 2.0.1 is generally available

 

All,

 

We are pleased to announce that DAOS 2.0.1 release is now generally available. This maintenance release contains the following updates on top of DAOS 2.0.0:

 

  • DAOS 2.0.1 includes fixes to the EC, VOS and Object services, as well as improvements to the control system and dfuse. It also includes numerous updates to the test and build infrastructure.
  • log4j-core has been updated from 2.16.0 to 2.17.1 DAOS-8929.
  • libfabric has been updated from 1.14.0~rc3-2 to 1.14.0-1. This also fixes the DAOS 2.0.0 known limitation with MOFED > 5.4-1.0.3.0 described in DAOS-9376.
  • mercury has been updated from 2.1.0~rc4-1 to 2.1.0~rc4-3. This fixes the high CPU utilization issue in DAOS 2.0.0 described in DAOS-9325
  • spdk has been updated from 21.07-8 to 21.07-11 (minor fixes only).

 

There are a number of resources available for the release:

As always, feel free to use this mailing list for any issues you may find with the release, or our JIRA bug tracking system, available at https://daosio.atlassian.net/jira, or on our Slack channel, available at https://daos-stack.slack.com.

 

Regards,

 

Kelsey Prantis

Senior Software Engineering Manager

Extreme Storage Architecture and Development Division

Intel

 

 


Announcement: DAOS 2.0.1 is generally available

Prantis, Kelsey
 

All,

 

We are pleased to announce that DAOS 2.0.2 release is now generally available. This maintenance release contains the following updates on top of DAOS 2.0.0:

 

  • DAOS 2.0.1 includes fixes to the EC, VOS and Object services, as well as improvements to the control system and dfuse. It also includes numerous updates to the test and build infrastructure.
  • log4j-core has been updated from 2.16.0 to 2.17.1 DAOS-8929.
  • libfabric has been updated from 1.14.0~rc3-2 to 1.14.0-1. This also fixes the DAOS 2.0.0 known limitation with MOFED > 5.4-1.0.3.0 described in DAOS-9376.
  • mercury has been updated from 2.1.0~rc4-1 to 2.1.0~rc4-3. This fixes the high CPU utilization issue in DAOS 2.0.0 described in DAOS-9325
  • spdk has been updated from 21.07-8 to 21.07-11 (minor fixes only).

 

There are a number of resources available for the release:

As always, feel free to use this mailing list for any issues you may find with the release, or our JIRA bug tracking system, available at https://daosio.atlassian.net/jira, or on our Slack channel, available at https://daos-stack.slack.com.

 

Regards,

 

Kelsey Prantis

Senior Software Engineering Manager

Extreme Storage Architecture and Development Division

Intel

 

 


Re: Consistency problem in daos

Lombardi, Johann
 

Hi there,

 

If abort is called either by the application directly (daos_tx_abort()) or by DAOS internally (something went wrong during commit), then all changes associated with the transaction are discarded and consistency is preserved. None of the uncommitted modifications are visible to external readers.

 

Cheers,

Johann

 

From: <daos@daos.groups.io> on behalf of "stephen.tao@..." <stephen.tao@...>
Reply-To: "daos@daos.groups.io" <daos@daos.groups.io>
Date: Saturday 29 January 2022 at 10:47
To: "daos@daos.groups.io" <daos@daos.groups.io>
Subject: [daos] Consistency problem in daos

 

1. Will abort be triggered if a transaction fails in the commit phase in Daos? (I read the code myself and found that this happens when the submission fails. At this time, some nodes may submit successfully)
2. If abort is triggered, what is the meaning? Will strong consistency be lost?

---------------------------------------------------------------------
Intel Corporation SAS (French simplified joint stock company)
Registered headquarters: "Les Montalets"- 2, rue de Paris,
92196 Meudon Cedex, France
Registration Number:  302 456 199 R.C.S. NANTERRE
Capital: 4,572,000 Euros

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.


Consistency problem in daos

@stephen.tao
 

1. Will abort be triggered if a transaction fails in the commit phase in Daos? (I read the code myself and found that this happens when the submission fails. At this time, some nodes may submit successfully)
2. If abort is triggered, what is the meaning? Will strong consistency be lost?