Hi,

Could you please advise what docker command you ran?

“docker build https://github.com/daos-stack/daos.git#master:utils/docker -f Dockerfile.centos.7” should work fine.

Regarding the REST API, it would be great to know the type of information you would like to expose via your REST interface.

Cheers,

Johann

I am trying to Set-up the DAOS system on my local machine. So, when I am trying to run the DockerFile for the same, it gives me the following error:



Also, I want to connect my REST API to the DAOS Server, and I know there is something called 'client APIs ' for DAOS. But, I need to know how to integrate my REST API with DAOS. 
It would be great if anyone could help me with this.

This will create a build a docker image using the Dockerfile from github, but the sources from the current directory so you need to do this from a checked-out copy of the source tree.

Ashley.

Hi,

Could you please advise what docker command you ran?

“docker build https://github.com/daos-stack/daos.git#master:utils/docker -f Dockerfile.centos.7” should work fine.

Regarding the REST API, it would be great to know the type of information you would like to expose via your REST interface.

Cheers,

Johann

I am trying to Set-up the DAOS system on my local machine. So, when I am trying to run the DockerFile for the same, it gives me the following error:



Also, I want to connect my REST API to the DAOS Server, and I know there is something called 'client APIs ' for DAOS. But, I need to know how to integrate my REST API with DAOS. 
It would be great if anyone could help me with this.

---------------------------------------------------------------------
Intel Corporation SAS (French simplified joint stock company)
Registered headquarters: "Les Montalets"- 2, rue de Paris,
92196 Meudon Cedex, France
Registration Number:  302 456 199 R.C.S. NANTERRE
Capital: 4,572,000 Euros

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

It looks like you’re trying to use 1.0.1?  I’ve just checked that out locally and there’s no reference to patchelf at all in the source tree at that point, we use it now for checking that build versions are relocatable.

Is there any chance you’re using a 1.0.1 dockerfile to build from the master branch?  The line numbers would support this, in particular SConstruct didn’t have 463 lines in v1.0.1, but in master it does match the trace you posted below.

Ashley.

Hi Ashley,

Thank you so much for the response. I was in the wrong directory it seems. Although when I try now, the build stops with the following error :- 

[...]

at include/opa_user.h    /tmp/tmp.8clvxfJtXv/opa_user_gen1.h  > /tmp/tmp.8clvxfJtXv/opa_user.h

cat include/opa_service.h /tmp/tmp.8clvxfJtXv/opa_service_gen1.h > /tmp/tmp.8clvxfJtXv/opa_service.h

install -m 0644 -D /tmp/tmp.8clvxfJtXv/opa_user.h    /usr/prereq/release/psm2/include/hfi1diag/opa_user.h

install -m 0644 -D /tmp/tmp.8clvxfJtXv/opa_service.h /usr/prereq/release/psm2/include/hfi1diag/opa_service.h

install -m 0644 -D /tmp/tmp.8clvxfJtXv/opa_common_gen1.h /usr/prereq/release/psm2/include/hfi1diag/opa_common.h

install -m 0644 -D include/opa_byteorder.h /usr/prereq/release/psm2/include/hfi1diag/opa_byteorder.h

install -m 0644 -D include/psm2_mock_testing.h /usr/prereq/release/psm2/include/hfi1diag/psm2_mock_testing.h

install -m 0644 -D include/opa_revision.h /usr/prereq/release/psm2/include/hfi1diag/opa_revision.h

install -m 0644 -D psmi_wrappers.h /usr/prereq/release/psm2/include/hfi1diag/psmi_wrappers.h

install -m 0644 -D psm_hal_gen1/hfi1_deprecated_gen1.h /usr/prereq/release/psm2/include/hfi1diag/hfi1_deprecated.h

rm -fr /tmp/tmp.8clvxfJtXv

Checking for C header file psm2.h... yes

Checking for C library psm2... yes

Checking whether patchelf program exists...no

MissingSystemLibs: ofi has unmet dependencies required for build:

  File "/home/daos/daos/SConstruct", line 463:

    scons()

  File "/home/daos/daos/SConstruct", line 386:

    preload_prereqs(prereqs)

  File "/home/daos/daos/SConstruct", line 175:

    prereqs.load_definitions(prebuild=reqs)

  File "/home/daos/daos/utils/sl/prereq_tools/base.py", line 992:

    self.require(env, comp)

  File "/home/daos/daos/utils/sl/prereq_tools/base.py", line 1061:

    raise error

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Hi,

Sorry for the confusion. The structure of the docker files was modified some time ago and we updated the documentation (in the source code too) accordingly, but the online github I/O pages have not been refreshed since. This explains the disconnection. Please run instead:

$ docker build https://github.com/daos-stack/daos.git#master -f utils/docker/Dockerfile.centos.7 -t daos

I have just tested it and it works.

@Durfey, Craig, could you please refresh the online documentation?

Regarding the management API, it is written in Go. For unknown reason, pkg.go.dev does not seem to be generating the documentation for our API any longer. I will get back to you ASAP on this.

Cheers,

Johann

Hi Johann,
I ran the exact same command: docker build https://github.com/daos-stack/daos.git#master:utils/docker \ -f Dockerfile.centos.7 -t daos

My major use case of connecting my Python API with the DAOS storage is to access the "management API". I know with pydaos we can read and write KV pairs to store with keys and value both to be strings. But is there any way to perform management tasks from my rest APi by interacting with the DAOS system. 

Ok, I’ve reproduced this.  The issue is the dockerfile in 1.0.1 is doing a git clone rather than a COPY to get the code, so the dockerfile is fetching the wrong data.

I guess the fix would be to update the Dockfile on the branch to have it pull from the branch, however as  Johann said we’ve modified the dockefiles a lot since then, and will be doing so again shortly.

The way forward is probably to checkout the code, locally modify the dockerfile to run “COPY ./ .” rather than the git clone command if you have the option to do this.

On master specifying the URL of a dockerfile on the command line isn’t going to work anyway, if we do want to support this method of install then we’ll need to rethink a bit.

Ashley.

Hi Ashley,
I ran the following 3 commands: 
git clone --recurse-submodules -b v1.0.1 https://github.com/daos-stack/daos.git
cd daos

Well, the URL on master definitely works if you use “docker build https://github.com/daos-stack/daos.git#master -f utils/docker/Dockerfile.centos.7”

Johann

Ok, I’ve reproduced this.  The issue is the dockerfile in 1.0.1 is doing a git clone rather than a COPY to get the code, so the dockerfile is fetching the wrong data.

I guess the fix would be to update the Dockfile on the branch to have it pull from the branch, however as  Johann said we’ve modified the dockefiles a lot since then, and will be doing so again shortly.

The way forward is probably to checkout the code, locally modify the dockerfile to run “COPY ./ .” rather than the git clone command if you have the option to do this.

On master specifying the URL of a dockerfile on the command line isn’t going to work anyway, if we do want to support this method of install then we’ll need to rethink a bit.

Ashley.

Hi Ashley,
I ran the following 3 commands: 
git clone --recurse-submodules -b v1.0.1 https://github.com/daos-stack/daos.git
cd daos

---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

Hi,

The dmg format command actually connects to the server to initiate the format. In your case, the server failed to start, so the “refused connection” is expected. I assume that you are running the server container with --privileged --cap-add=ALL -v /dev:/dev as specified in the documentation, right?

Cheers,

Johann

Hi Johann,

Thank you for all the help. The master definitely worked. And I was able to get a container running. Upon running: sudo docker exec server daos_server start -o /home/daos/daos/utils/config/examples/daos_server_local.yml to start the server. I get the following errors: 

DAOS Server config loaded from /home/daos/daos/utils/config/examples/daos_server_local.yml

daos_server logging to file /tmp/daos_control.log

ERROR: server: code = 647 description = "requested 4096 hugepages; got 0"

ERROR: server: code = 647 resolution = "reboot the system or manually clear /dev/hugepages as appropriate"

I read on an older mail that these errors can be taken as warnings and can be ignored, the server would still run. is that right?
So I went ahead and tried to run: $ docker exec server dmg -i storage format.
But this command gives the following error: the server at localhost:10001 refused the connection

My understanding was if I use the daos_server_local.yml file, i do not need authentication as the config file sets the flag allow_insecure :true 

My target is to just have a basic daos server running and be able to use management API on it. so that I can integrate my client application to interact with the daos server using the management API

If NVMe is not required then try setting "nr_hugepages: 0" in the global section of the server configuration file.

Regards,

Tom