Re: Attempts at using PyDAOS


Pittman, Ashley M
 

 

Alex,

 

Can you elaborate on what it is you’re trying to achieve?  There are really two parts to the python bindings, there’s pydaos.raw which is using ctypes to open the daos client shared library from python, this is mostly used in the testing to create containers and do any setup for testing, or simply as a quick way to invoke the DAOS API.

 

The second part of the python binding is pydaos itself, which is a python module which is in part written in C and is a direct client of daos, it’s here that we’re concentrating most of the work on DAOS clients (as opposed to tools/setup) and this is where the key/value store is accessed.  For this you should look at the KVObj() object, which exports a dict like object, which can also support bulk put/get.  Also included but scheduled for overhaul and partial removal is the daosdbm module which is in the source tree but not installed, the daos_named_kv() object here is the one to look at for one way to import daos and open multiple kv stores by name.

 

https://github.com/daos-stack/daos/blob/master/src/client/pydaos/pydaos_core.py#L235

https://github.com/daos-stack/daos/blob/master/src/client/pydaos/daosdbm.py#L30-L72

 

Nothing has changed in this area in the last month, but there is work to change how daos_init() is called from within pydaos which will affect and hopefully improve the way the module is imported, but that’s still work-in-progress.

 

I’ll file a ticket and resolve the spelling errors in the codebase/documentation..

 

Ashley,

 

From: <daos@daos.groups.io> on behalf of "Alex Barcelo via groups.io" <alex@...>
Reply to: "daos@daos.groups.io" <daos@daos.groups.io>
Date: Tuesday, 5 May 2020 at 10:16
To: "daos@daos.groups.io" <daos@daos.groups.io>
Subject: [daos] Attempts at using PyDAOS

 

I have a DAOS installation for evaluation/exploration and I was trying to make the Python bindings work. My first contact with PyDAOS is at the official documentation https://daos-stack.github.io/user/interface/#python-bindings which talks about a "DAOS key-value store API with python dictionnaries[sic]" but doesn't give any example about how to create that object (or am I missing something obvious?).

I decided to move to something more elemental and play around the pydaos.raw bindings, seeing that there is some example here:

https://github.com/daos-stack/daos/blob/master/src/client/pydaos/raw/README.md

The imports are missing but are easy enough to guess (import json, import os, from pydaos.raw import <things that are used in the example>, etc.). To build the context I have to use /lib64 instead, i.e.:

    context = DaosContext(data['PREFIX'] + '/lib64/')  # instead of /lib

Otherwise I get a <daos_path>/install/lib/libdaos.so.: cannot open shared object file: No such file or directory. But that is fixed with that change (I am in a one-month old master branch, so apologies if that has already been fixed... I am unable to update and rebuild everything right now). I also had to do an explicit export OFI_INTERFACE=eno1 otherwise the import pydaos was failing with a Failed to initialize DAOS: DER_INVAL, but logs where helpful on that matter.

The problem I have not been able to solve is the following: When I reach the pool.create command, I get:

DaosApiError: Pool create returned non-zero. RC: -1026

I know that the problem is related to my non-root installation where instead of using /var/run/daos* I am using another folder (I figured that much in the logs with the line daos ERR src/common/drpc.c:173 unixcomm_connect() Failed to connect to /var/run/daos_agent/agent.sock, errno=2(No such file or directory)), but I recall having it correctly set in the daos_agent.yml and I have been able to create pools (no errors) with the dmg pool create command.

Of course I am missing some configuration step. Which configuration step am I missing? Where should I configure the non-default /var/run/daos_agent alternative path? Or am I doing something inappropriate?

---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

Join daos@daos.groups.io to automatically receive all group messages.