Hi there,

Please find below the DAOS community newsletter for September 2022. A copy of this newsletter is also available on the wiki.

Past Events

• Flash Memory Summit’22: 3^rd Workshop on Extreme-Scale Storage and Analysis (August 2^nd-4^th)
  Requirements and Challenges Associated with the World's Fastest Storage Platform
  https://www.flashmemorysummit.com
  Jeff Olivier (Intel)

Upcoming Events

• IXPUG Annual Conference 2022 (Sep 29)
  The Evolution of Storage and Memory and the DAOS Role in It
  Kevin Harms (ANL)
  Andrey Kudryavtsev (Intel)
• SuperCheck-SC'22 (Nov 14)
  DAOS: Nextgen Storage Stack for HPC and AI
  Johann Lombardi (Intel)
• SC'22 BoF (Nov 15-17)
  DAOS Storage Community BoF
  Kevin Harms (ANL)
  Michael Hennecke (Intel)
  Dean Hildebrand (Google)
  Panagiotis Adamidis (DKRZ)
• SC'22 BoF (Nov 15-17)
  The Storage Tower of Babel? ... Not! Actually, maybe?
  Philippe Deniel (CEA)
  John Bent (Seagate)
  Tiago Quintino (ECMWF)
  Johann Lombardi (Intel)
• SC'22 Tutorial (Nov 13-14)
  Emerging Storage Interfaces: DAOS and PMDK
  Adrian Jackson (EPCC)
  Mohamad Chaarawi (Intel)
  Johann Lombardi (Intel) 
• 6th annual DAOS User Group (Nov/Dec'22)

Release

• Current stable release is 2.0.3. See https://docs.daos.io/v2.0/ and https://packages.daos.io/v2.0/ for more information.
  2.0.3 includes several fixes for ARM64 support, erasure code and pool operations. Please see the release notes for more details.
• Branches:
• release/2.0 is the release branch for the stable 2.0 release. Latest bug fix release is 2.0.3 (v2.0.3 tag).
• release/2.2 is the development branch for the future 2.2 release. The first release candidate has been created (v2.2.0-rc1 tag).
• Master is the development branch for the future 2.4 release. Latest test build is 2.3.100 (v2.3.100-tb tag). New build including EC parity rotation feature imminent.
• Major recent changes on release/2.0 (bugfix release):
• Several coverty fixes
• Fix incorrect assertion failure hit when running soak testing with LAMMPS application
• Bump hadoop-common version to 3.3.3
• Several documentation fixes
• Several test fixes.
• Major recent changes on release/2.2 (future 2.2 release):
• All patches listed in the 2.0 section above.
• Update mercury to 2.2.0
• Update pmdk to 1.12.1
• Trigger DTX reindex before DTX resync
• Fix issue with srx_disabled config field
• Fix mtime set to not rely on DAOS HLC
• Improve DAOS build preprocessing steps
• Fix java jar build instructions
• Reduce lock contention on hash lock in libdaos to increase multi-thread performance
• Set UCX_IB_FORK_INIT env var in the engine
• Add new metrics to track EC full stripe and partial updates
• Improve dfs_setattr to re-sample mtime on file size changes
• Add UCX documentation
• Do not use stable epoch for reclaim
• Fix dfs_open for directories without O_EXCL
• Add support for 2.0/2.2 agent interoperability
• Major recent changes on master (future 2.4 release):
• All patches listed in the 2.2 section above.
• Add prefix to notice logging in the control plane
• Add githook install script
• Move NLT and unit tests to el8
• Fix a race in dc_tx_get_epoch
• Fix name match in daos_oclass_name2id()
• Add ability for engine to manage its own ABT stack via mmap() to pro-actively detect stack overrun
• Limit number of outstanding I/Os to NVMe device
• Remove indirect link for ISA-L
• Store scan objects target ID during rebuild to avoid excessive iteration when sending object list
• Create a single bulk handle per DMA chunk and share the same handle for all bulk transfer against the same DMA chunk.
• Retry map_fresh on more errors
• Refactor daos_server standalone command surface
• Reject read/write hole in bio
• Run NLT on ARM64 self-hosted runners
• Fix gap in EC rotation patch in tx classify
• Replace SWIM D_CIRCLEQ with a hash table.
• Fix VMD domain parsing
• Accept positional args in dfuse command to support mtab entries
• Set EC cell alignment to 32 bytes
• Disallow IP address with negative port in the control plane
• What is coming:
• 2.2.0 GA
• 2.4.0 feature freeze

R&D

• Major features under development:
• VOS on SPDK blob
• Detailed design documented here Metadata on SSDs including the WAL layout (VOS blob and WAL blob layout)
• All subtasks created under DAOS-11040 and story points added.
• Impact on performance still under evaluation.
• Branch: feature/vos-on-blob
• Target release: TBD
• Multi-user dfuse
• Allow dfuse mountpoint to be accessible by multiple users
• Still working on extending test coverage before landing the patch on master.
• PR: https://github.com/daos-stack/daos/pull/3956
• Target release: 2.4
• More aggressive caching in dfuse for AI APPs
• FUSE version updated for EL8 for readdir caching support, not needed on Leap that was recent enough FUSE version.
• FUSE kernel readdir is on enabled, dfuse readdir still under work.
• PR: https://github.com/daos-stack/daos/pull/6776
• Target release: 2.4
• Catastrophic recovery
• Aka distributed fsck or checker
• Tests for ddb (low level debugger utility similar to debugfs for ext4) under review
• Testing for the dmg checker under development
• Pass 4 for container recovery completed.
• Branch: feature/cat_recovery
• Target release: 2.6
• Multi-homed network support
• Aka multi-provider support
• This feature aims at supporting multiple network provider in the engine
• Branch is feature complete now and testing is underway
• Branch: feature/multiprovider
• Target release: 2.6
• Client-side metrics
• API to collect libdaos metrics to be eventually integrated with Darshan
• PR: https://github.com/daos-stack/daos/pull/6497
• Target release: 2.6
• Performance domain
• Extend placement algorithm to be aware of fabric topology
• Fix to avoid putting shards on the same domain landed
• Branch: feature/perf_dom
• Target release: 2.8 
• Pathfinding:
• DAOS Pipeline API for active storage
• Feature branch created and addressing test failure
• MariaDB DAOS engine with predicate pushdown: https://github.com/daos-stack/mariadb/tree/daos_edu
• Prototype of server-side find using pipeline API
• Branch: feature/pipeline_api
• Leveraging the Intel Data Streaming Accelerator (DSA) to accelerate DAOS
• Prototype leveraging DSA for VOS aggregation delivered
• Initial results shared at IXPUG conference.
• OPX provider support in collaboration with Cornelis Networks
• OPX provider merged upstream in libfabric
• Provider supported in latest mercury version
• Changes to DAOS to enable OPX as part of the build in progress
• GPU data path optimizations
• I/O Middleware / Framework Support
• TensorFlow-IO plugin for DAOS
• https://github.com/daos-stack/tensorflow-io-daos/blob/devel/docs/daos_tf_docs.md
• https://docs.daos.io/v2.2/user/tensorflow/
• Upstream PR has been discussed with maintainer and should be hopefully merged soon.
• S3 support via a DAOS backend to Rados Gateway (RGW)
• Migration to libds3 (new library in DAOS to implement S3 interface over DAOS) completed
• libds3 under consideration for integration on master.
• PR submitted upstream by Zuhair/Seagate and expected to be merged soon: https://github.com/ceph/ceph/pull/47709
• Block interface over DAOS using SPDK DAOS bdev
• PR submitted upstream by DenisB/Croit has been merged upstream! https://github.com/spdk/spdk/commit/2e283fcb67a8ee1d9b4f470f17bec57bbe3adad5
• DAOS SPDK bdev is thus part of the upcoming SPDK 22.09 release.
• This entry will be removed from this report next time.
• DAOS/DFS integration with NFS Ganesha
• Still under exploration by Croit

News

• In addition to building on ARM platform on Ubuntu 22.04, AlmaLinux 8 and Leap 15, some basic tests (called NLT, stands for Node Local Tests) are now run on every PR landing. See this link for more information .Thanks again for Linaro and Croit for their support.Next step is to run unit tests.
• Congrats to Croit and DenisB for merging the SPDK DAOS bdev upstream!
• The  DAOS community BoF for SC'22 has been accepted!