Re: DAOS with NVMe-over-Fabrics


Nabarro, Tom
 

In order to get DAOS up and running with NVMe-oF I think some internal changes in bio might be needed.

 

Does the use of NVMe-oF provide specific benefits in a DAOS installation? Would like to understand the incentives behind using NVMe-oF with DAOS.

 

Command/log output would also be useful.

Regards,

Tom Nabarro – DCG/ESAD

M: +44 (0)7786 260986

Skype: tom.nabarro

 

From: daos@daos.groups.io <daos@daos.groups.io> On Behalf Of Lombardi, Johann
Sent: Friday, October 9, 2020 7:02 AM
To: daos@daos.groups.io
Subject: Re: [daos] DAOS with NVMe-over-Fabrics

 

Hi Anton,

 

The DAOS control plane does not support NVMe-oF target yet, that’s why you don’t see them via storage scan. That being said, my hope was that, by changing the daos_nvme.conf manually as you did, you would be able you to start the I/O server with NVMe-oF targets. Maybe some glue is still missing.

 

Cheers,

Johann

 

From: <daos@daos.groups.io> on behalf of "anton.brekhov@..." <anton.brekhov@...>
Reply-To: "daos@daos.groups.io" <daos@daos.groups.io>
Date: Wednesday 7 October 2020 at 16:09
To: "daos@daos.groups.io" <daos@daos.groups.io>
Subject: Re: [daos] DAOS with NVMe-over-Fabrics

 

On Thu, Sep 17, 2020 at 12:56 AM, Lombardi, Johann wrote:

adrfam:IPv4 traddr:10.9.1.118 trsvcid:4420 subnqn:test

I've tried to change daos_nvme.conf in runtime of daos server and before starting due to connect disk through rdma. In both ways I cannot see it in daos_system. nvme discover see exported disk.

When SPDK taking this disk in system? Or should I write this in other files? My daos_nvme.conf
[Nvme]

    TransportID "trtype:PCIe traddr:0000:b1:00.0" Nvme_apache512_0

    TransportID "trtype:PCIe traddr:0000:b2:00.0" Nvme_apache512_1

    TransportID "trtype:PCIe traddr:0000:b3:00.0" Nvme_apache512_2

    TransportID "trtype:PCIe traddr:0000:b4:00.0" Nvme_apache512_3

    TransportID "trtype:rdma adrfam:IPv4 traddr:10.0.1.2 trsvcid:4420 subnqn:nvme-subsystem-name" Nvme_apache512_4

    RetryCount 4

    TimeoutUsec 0

    ActionOnTimeout None

    AdminPollRate 100000

    HotplugEnable No

    HotplugPollRate 0

And nvme discover output:

[root@apache512 ~]# nvme discover -t rdma -a 10.0.1.2 -s 4420

 

Discovery Log Number of Records 1, Generation counter 2

=====Discovery Log Entry 0======

trtype:  rdma

adrfam:  ipv4

subtype: nvme subsystem

treq:    not specified, sq flow control disable supported

portid:  1

trsvcid: 4420

subnqn:  nvme-subsystem-name

traddr:  10.0.1.2

rdma_prtype: not specified

rdma_qptype: connected

rdma_cms:    rdma-cm

 

rdma_pkey: 0x0000

---------------------------------------------------------------------
Intel Corporation SAS (French simplified joint stock company)
Registered headquarters: "Les Montalets"- 2, rue de Paris,
92196 Meudon Cedex, France
Registration Number:  302 456 199 R.C.S. NANTERRE
Capital: 4,572,000 Euros

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

Join daos@daos.groups.io to automatically receive all group messages.