DAOS Community Update / Oct'22

Lombardi, Johann

Hi there,

Please find below the DAOS community newsletter for October 2022. A copy of this newsletter is also available on the wiki.

Past Events

Upcoming Events


  • Current stable release is 2.2.0 released on Oct 21. See https://docs.daos.io/v2.2/ and https://packages.daos.io/v2.2/ for more information. Please see the release notes for more details.
  • With the release of 2.2.0, 2.0.x releases are declared end-of-life.
  • Branches:
    • release/2.2 is the release branch for the stable 2.2 release. Latest bug fix release is 2.2.0 (v2.2.0 tag).
    • Master is the development branch for the future 2.4 release. Latest test build is 2.3.101 (v2.3.101-tb tag) including the EC rotation feature.
  • Major recent changes on release/2.2 (future 2.2 release):
    • Fix VMD domain parsing
    • Fix PS replica leaks
    • Fix 2.0/2.2 interoperability issue with pool RF
    • Fix assertion failure in dc_cont_free()
    • Fix race condition in cart
    • Address memory corruption during key_query
    • Several fixes for EC migration
    • Check and reset NONEXIST in iter_next and probe
    • Bump protobuf-java from 3.16.1 to 3.16.3
  • Major recent changes on master (future 2.4 release):
    • All patches listed in the 2.2 section above.
    • Fix a bug in key enumeration associated with ads[0].kd_key_len
    • Add support for rf_lvl to cont create api on pydaos
    • Enable EC parity rotation by default
    • Add missing void in dfs_init/fini declaration
    • Remove RPC post increment restriction preventing extra RPC handles from being posted upon exhaustion
    • Re-enable custom RPC timeout in RDB
    • Remove ability to build w/o stdatomic.h
    • Add bulk and vos latency to metrics
    • Skip reclaim job during merge
    • Fix some DTX visibility issues
    • Allo daos_server network scan to run w/o config
    • Update DAOS to use UCX 1.13 and disable UCX multi-rail support
    • Don't hold lock for d_hhash_link_get/putref
    • Add dmg system exclude
    • Fix auto object class selection for RP hints for arrays
    • Don't set pool destroy state if service is not up
    • Improve PS reconfigurations
    • Add IOPS info to daos pool autotest
    • Fix swim paranoia
    • Reject invalid number of pool create ranks
    • Add config option to agent to ignore interfaces
    • Several fixes to EC parity rotation
    • Add support for pull request template
    • Fix a number of python flake issues
    • Add ability to run server under valgrind
    • Add NUMA affinity to tmpfs mount options
    • Add pool svc list to property query
    • Bypass checks in pool evict rdb tx update
    • Several IV fixes
    • Remove CentOS7 leftovers
    • Add DFS readdirplus API
    • Several checksum scrubbing upgrade fixes
    • Rename privileged helper from daos_admin to daos_server_helper
    • Rename rf and rf_level properties to rd_fac and rd_lvl
    • Add rebuild version to pool query
    • Bump garbage collection ULT stack size
  • What is coming:
    • 2.2.1 bug fix release
    • 2.4.0 feature freeze


  • Major features under development:
    • VOS on SPDK blob
      • Detailed design documented here Metadata on SSDs including the WAL layout (Meta blob and WAL blob layout)
      • All development and testing tasks are tracked under DAOS-11040 for phase 1.
      • Changes to the yaml file implemented. WAL infrastructure and metadata blob creation landed.
      • PMDK-based allocator extracted and integrated into DAOS. Early performance evaluation in progress.
      • Branch: feature/vos-on-blob
      • Target release: 2.4 (phase 1 preview)
    • Multi-user dfuse
    • More aggressive caching in dfuse for AI APPs
      • FUSE version updated for EL8 for readdir caching support, not needed on Leap that was recent enough FUSE version.
      • FUSE kernel readdir is on enabled, dfuse readdir still under work.
      • PR: https://github.com/daos-stack/daos/pull/6776
      • Target release: 2.4
    • Catastrophic recovery
      • Aka distributed fsck or checker
      • Tests for ddb (low level debugger utility similar to debugfs for ext4) landed
      • Testing for the dmg checker landed.
      • Testing for pass 3 and 4 under development.
      • Pass 4 for container recovery completed.
      • Branch: feature/cat_recovery
      • Target release: 2.6
    • Multi-homed network support
      • Aka multi-provider support
      • This feature aims at supporting multiple network provider in the engine
      • Branch is feature complete now and testing is underway
      • Branch: feature/multiprovider
      • Target release: 2.6
    • Client-side metrics
    • Performance domain
      • Extend placement algorithm to be aware of fabric topology
      • Fix to avoid putting shards on the same domain landed
      • Branch: feature/perf_dom
      • Target release: 2.8
  • Pathfinding:
    • DAOS Pipeline API for active storage
    • Leveraging the Intel Data Streaming Accelerator (DSA) to accelerate DAOS
      • Prototype leveraging DSA for VOS aggregation delivered
      • Initial results shared at IXPUG conference.
    • OPX provider support in collaboration with Cornelis Networks
      • OPX provider merged upstream in libfabric
      • Provider supported in latest mercury version
      • Changes to DAOS to enable OPX as part of the build in progress
    • GPU data path optimizations
  • I/O Middleware / Framework Support:


  • Congratulation to the Seagate team for the integration of the DAOS backend to the Rados Gateway (RGW)!
  • Updated DAOS roadmap including changes for the md_on_ssd phase 1 and phase 2 project to be available soon.


Intel Corporation SAS (French simplified joint stock company)
Registered headquarters: "Les Montalets"- 2, rue de Paris,
92196 Meudon Cedex, France
Registration Number:  302 456 199 R.C.S. NANTERRE
Capital: 5 208 026.16 Euros

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.