Alex,

Can you elaborate on what it is you’re trying to achieve?  There are really two parts to the python bindings, there’s pydaos.raw which is using ctypes to open the daos client shared library from python, this is mostly used in the testing to create containers and do any setup for testing, or simply as a quick way to invoke the DAOS API.

The second part of the python binding is pydaos itself, which is a python module which is in part written in C and is a direct client of daos, it’s here that we’re concentrating most of the work on DAOS clients (as opposed to tools/setup) and this is where the key/value store is accessed.  For this you should look at the KVObj() object, which exports a dict like object, which can also support bulk put/get.  Also included but scheduled for overhaul and partial removal is the daosdbm module which is in the source tree but not installed, the daos_named_kv() object here is the one to look at for one way to import daos and open multiple kv stores by name.

https://github.com/daos-stack/daos/blob/master/src/client/pydaos/pydaos_core.py#L235

https://github.com/daos-stack/daos/blob/master/src/client/pydaos/daosdbm.py#L30-L72

Nothing has changed in this area in the last month, but there is work to change how daos_init() is called from within pydaos which will affect and hopefully improve the way the module is imported, but that’s still work-in-progress.

I’ll file a ticket and resolve the spelling errors in the codebase/documentation..

Ashley,

I have a DAOS installation for evaluation/exploration and I was trying to make the Python bindings work. My first contact with PyDAOS is at the official documentation https://daos-stack.github.io/user/interface/#python-bindings which talks about a "DAOS key-value store API with python dictionnaries[sic]" but doesn't give any example about how to create that object (or am I missing something obvious?).

I decided to move to something more elemental and play around the pydaos.raw bindings, seeing that there is some example here:

https://github.com/daos-stack/daos/blob/master/src/client/pydaos/raw/README.md

The imports are missing but are easy enough to guess (import json, import os, from pydaos.raw import <things that are used in the example>, etc.). To build the context I have to use /lib64 instead, i.e.:

    context = DaosContext(data['PREFIX'] + '/lib64/')  # instead of /lib

Otherwise I get a <daos_path>/install/lib/libdaos.so.: cannot open shared object file: No such file or directory. But that is fixed with that change (I am in a one-month old master branch, so apologies if that has already been fixed... I am unable to update and rebuild everything right now). I also had to do an explicit export OFI_INTERFACE=eno1 otherwise the import pydaos was failing with a Failed to initialize DAOS: DER_INVAL, but logs where helpful on that matter.

The problem I have not been able to solve is the following: When I reach the pool.create command, I get:

DaosApiError: Pool create returned non-zero. RC: -1026

I know that the problem is related to my non-root installation where instead of using /var/run/daos* I am using another folder (I figured that much in the logs with the line daos ERR src/common/drpc.c:173 unixcomm_connect() Failed to connect to /var/run/daos_agent/agent.sock, errno=2(No such file or directory)), but I recall having it correctly set in the daos_agent.yml and I have been able to create pools (no errors) with the dmg pool create command.

Of course I am missing some configuration step. Which configuration step am I missing? Where should I configure the non-default /var/run/daos_agent alternative path? Or am I doing something inappropriate?

The easiest way, assuming you want a single KV per container is to do the following:

import pydaos

cont = pydaos.cont(‘<pool_uuid>’, ‘<container_uuid>’)

kv = cont.root_kv()

kv[‘key’] = value

If you want multiple kvs per container then you have to do something like daos_named_kv and store the name ó object_id translation in the root kv, or you can call cont.newkv() to create a new key-value store, and save kv.oid.lo and kv.oid.hi out-of-band, reopening the kv later on with kv = cont.kv(pydaos.OjbID(lo, hi).

The issues with the socket come from non-default install paths, but this is OK, you need to set DAOS_AGENT_DRPC_DIR to the directory that your agent was started with, as documented here https://daos-stack.github.io/admin/deployment/  I assume you have the agent and server already running?

Ashley.

Ok, regarding KVObj, my naive attempt was:

import pydaos

pydaos.KVObj()

And that obviously fails due to missing positional arguments. I didn't realize on the factory-like mechanisms through the daos_named_kv class. But I don't follow your steps, should I use daos_named_kv or I should not? I don't really see which parameter values should I use --I am trying to follow the single-server installation instructions as closely as possible, trying to remain in the default-land.

Even if I get the KV interface to work, wouldn't I have the same issues with the broken socket which I am already having while trying pydaos.raw? Or are there some key differences between both of them? Should I configure the /var/run/daos_agent path somewhere? Why can I do things like pool creation through CLI but the Python pydaos.raw break?

On the "what I am trying to achieve": I want to "validate" (for lack of a better word) that the installation of DAOS I have in our development machine is correct and is using properly its OptaneDC PMM --that is the only deviation from the single-server defaults I have consciously introduced in my installation so far. For now, I have been able to build DAOS in the machine and start all the DAOS services, but I haven't managed to use it from application space yet. And PyDAOS seemed a good way to proceed, is it?

That looks much better.  You should be aware that daos will be using a byte-array for the values, so in this case 42 will be a string, not a integer.  It’s likely that you’d want to use pickle/unpickle complex values before storing them.

Ashley.

Thanks, I didn't know about pydaos.Cont and was a bit lost.

I completely missed that section on the deployment documentation, shame on me. I was definetely looking at the wrong places, thanks for pointing that out! I have both the server and the agent running, and thanks to your pointers I have been able to create a key-value from Python and use it.

My success full steps:

$ dmg pool create -i --scm-size=40G --nvme-size=0G

localhost:10001: connected

Pool-create command SUCCEEDED: UUID: 7d8d7aeb-3157-41dd-84cc-286d08a9b8d5, Service replicas: 0

$ daos cont create --pool=7d8d7aeb-3157-41dd-84cc-286d08a9b8d5 --svc=0

Successfully created container 967ce7a8-87c2-4982-a771-d1cc88e490da

$ daos pool list-cont --svc=0 --pool=7d8d7aeb-3157-41dd-84cc-286d08a9b8d5

 05/06-10:33:55.66 Apass 967ce7a8-87c2-4982-a771-d1cc88e490da

$ python

>>> import pydaos

>>> pydaos.Cont('7d8d7aeb-3157-41dd-84cc-286d08a9b8d5', '967ce7a8-87c2-4982-a771-d1cc88e490da')

<pydaos.pydaos_core.Cont object at 0x7f2eeff8f0b8>

>>> kv = cont.rootkv()

>>> kv['answer'] = '42'