Re: DAOS Distributed Transaction

Changwoo Min

Hi Johann,

Thank you for your reply. It helps a lot. 

Changwoo Min

On Tue, Sep 8, 2020 at 1:12 PM Lombardi, Johann <johann.lombardi@...> wrote:

Hi there!


First of all, thanks for joining the community. I am *very* excited about the support of distributed transactions in DAOS. We have been working on this for a long time and completely changed the design in 2017 to properly support serializability and external consistency.


The first use case is to maintain internal consistency of our POSIX layer called DAOS File System (DFS). The integration with distributed transactions allows to improve the POSIX compliance of the DFS library by guaranteeing atomicity of metadata operations like rename(2). No orphans or dandling entries are left behind in case of a client crashing in the middle of a rename operation.


Since POSIX is just yet-another middleware library for us, the same applies to all I/O middleware libraries built natively over DAOS.  With the HDF5 DAOS VOL, HDF5 datasets can be updated safely in place without any risk of corrupting the internal HDF5 data structures when the application crashes or quits unexpectedly. One HDF5 operation typically requires multiple KV fetches/updates over the DAOS case. By bundling all those low-level operations under a single DAOS transaction, we can guarantee that the high-level HDF5 operation is atomic. Since the HDF5 DAOS VOL also supports independent operations (e.g. concurrent non-collective HDF5 group creations), the use of DAOS transactions also allow to preserve internal consistency when processing concurrent uncoordinated HDF5 operations. It is used in a similar way in several domain-specific data models that are in the process of being ported to the native DAOS API, bypassing POSIX entirely.


Last but not least, distributed transactions allow to support database semantics directly over DAOS. We are actually in the process of porting a SQL engine over DAOS as a proof of concept. I do believe that this capability combined with computational storage (i.e. running simple data-intensive tasks on DAOS storage nodes directly w/o moving the data over the fabric) will open the door to many interesting opportunities (e.g. query/indexing, …) in areas like data analytics and AI.





From: <> on behalf of Changwoo Min <changwoo@...>
Reply-To: "" <>
Date: Saturday 5 September 2020 at 06:54
To: "" <>
Subject: [daos] DAOS Distributed Transaction


Hi DAOS community!

I'm Changwoo Min, a professor at Virginia Tech. My group does research on persistent memory and storage systems. I found that DAOS is an exciting and cool project!

In particular, it is interesting to me that DAOS supports distributed transactions. I am wondering what the typical/intended use cases and applications of the distributed transaction are. Especially considering, as far as I know, DAOS will be deployed to HPC systems. I wonder if transactions can benefit any HPC/AI/ML/analytics applications. Any comments will be helpful.

Changwoo Min

Intel Corporation SAS (French simplified joint stock company)
Registered headquarters: "Les Montalets"- 2, rue de Paris,
92196 Meudon Cedex, France
Registration Number:  302 456 199 R.C.S. NANTERRE
Capital: 4,572,000 Euros

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

Join to automatically receive all group messages.