Hi there,

Please find below the DAOS community newsletter for December 2021.

Past Events (November)

• SC’21 Tutorial (Nov 15)
  Practical Persistent Memory Programming: PMDK and DAOS
  Adrian Jackson (University of Edinburgh)
  Mohamad Chaarawi (Intel)
  Johann Lombardi (Intel)
  https://sc21.supercomputing.org/presentation/?id=tut134&sess=sess210
• SC’21 BoF (Nov 16)
  Object-stores for HPC - a Devonian Explosion or an Extinction Event?
  Philippe Deniel, CEA
  John Bent, Seagate
  Tiago Quinto, ECMWF
  Johann Lombardi, Intel
  https://sc21.supercomputing.org/presentation/?id=bof122&sess=sess368
• DAOS User Group (DUG’21)
  Slide decks and recordings available at http://dug.daos.io
• Intel SC21 Booth
  Dev led talk:
  DAOS Unleashes the Power in HPC Applications: QCT(Quanta) DevCloud Experience Sharing
  A Compact, Scalable, Efficient Data Collection Solution for Edge/IoT to Cloud by Zettar
  Fireside chat:
  Cambridge Service for Data Driven Discovery Enables Real-time Hospital Decision Support Systems to Improve Outcomes
  https://hpcevents.intel.com/devhub

Upcoming Events

• None this month. 

Release

• 2.0.0 RC2 was tagged (v2.0.0-rc2) earlier this week.
• The only difference with 2.0.0 RC1 is the log4j CVE update in the Hadoop connector (not included in the RPMs).
• Master is the development branch for the future 2.2 release.
• Major recent 2.0 changes:
• Integrate libfabric fix for fi_cancel()
• Bump log4j-core version to 2.16.0 in the Hadoop connector
• Fix a race between EC aggregation and reintegrate operation exposed by soak testing
• Many documentation updates that will be on http://docs.daos.io soon
• Add new recipes and scripts to install RPMs and run DAOS in docker containers
• Fix a memory registration issue with verbs & rebuild
• Quiet group version mismatch errors at pool creation
• Fix a SWIM regression introduced recently
• Fix a bulk leak on error path
• Refactor DAOS object ID to encode the number of groups/shards and improve performance
• Add interception for openat
• Lock memory only for VOS pools instead of using mlockall
• Major recent master changes:
• Add new logic for punch propagation to accelerate empty directory removal
• Restructure usage of hwloc in the control plane
• Add CXI support to netdetect
• Several VMD-related fixes
• Rework how we use GitHub actions
• Add new interface to create (and track) a ABT ULT (sched_create_ult())
• Several DTX enhancements
• What is coming:
• rc2 validation

R&D

• Major features under development:
• Checksum scrubbing
• LDMS plugin to export DAOS metrics (targeted for 2.2)
• API to collect libdaos metrics to be integrated with Darshan (targeted for 2.4)
• Multi-user dfuse (targeted for 2.2)
• More aggressive caching in dfuse for AI APPs (targeted for 2.2)
• TensorFlow-IO plugin for DFS
• https://github.com/daos-stack/tensorflow-io-daos/blob/devel/docs/daos_tf_docs.md
• Pathfinding:
• MariaDB DAOS engine with predicate pushdown to the DAOS storage nodes
• Prototyped DAOS MariaDB engine available here: https://github.com/daos-stack/mariadb
• PR for pipeline API (#6238)
• Work in progress to support pipeline API in the engine
• Leveraging the Intel Data Streaming Accelerator (DSA) to accelerate DAOS
• Prototype leveraging DSA for VOS aggregation delivered

News

• The documentation is migrating to use the “mike” plugin from MkDocs for multi-version support
• DAOS landing page now available on intel.com
• https://www.intel.com/content/www/us/en/high-performance-computing/daos.html
• Article on how to use PyDAOS from Eduardo and Eeheet
• https://www.intel.com/content/www/us/en/developer/articles/case-study/unlock-the-power-of-daos-in-python-with-pydaos.html
• Staffing:
• We welcome Cedric Koch-Hofer (France) and Lei Huang (TX, USA) to the Intel DAOS team. Cedric and Lei join us as full-stack engineers.
• We welcome Ray Dennis (NM, USA) to the Intel DAOS. Ray is joining our DevOps team.