Re: FIO Results & Running IO500


Peter
 

Johann, my configuration is as follows:


4 nodes, 4 engines in total (1 per node)
2 x 128 GB Optane per socket, 2 x sockets per node (DAOS is currently only using 1 socket per node)
(We also have 1x 1.5 TB NVMe drive / node, that we plan to eventually configure DAOS to use)
Nodes are using Mellanox EDR Infiniband
Cent OS 7.9, have tried various MPI distributions.

The yaml file (was generated via the auto conf)

***********
port: 10001
transport_config:
  allow_insecure: true
  server_name: server
  client_cert_dir: /etc/daos/certs/clients
  ca_cert: /etc/daos/certs/daosCA.crt
  cert: /etc/daos/certs/server.crt
  key: /etc/daos/certs/server.key
servers: []
engines:
- targets: 16
  nr_xs_helpers: 3
  first_core: 0
  name: daos_server
  socket_dir: /var/run/daos_server
  log_file: /tmp/daos_engine.0.log
  scm_mount: /mnt/daos0
  scm_class: dcpm
  scm_list:
  - /dev/pmem0
  bdev_class: nvme
  provider: ofi+verbs;ofi_rxm
  fabric_iface: ib0
  fabric_iface_port: 31416
  pinned_numa_node: 0
disable_vfio: false
disable_vmd: true
nr_hugepages: 0
set_hugepages: false
control_log_mask: INFO
control_log_file: /tmp/daos_server.log
helper_log_file: ""
firmware_helper_log_file: ""
recreate_superblocks: false
fault_path: ""
name: daos_server
socket_dir: /var/run/daos_server
provider: ofi+verbs;ofi_rxm
modules: ""
access_points:
- 172.23.7.3:10001
fault_cb: ""
hyperthreads: false
path: ../etc/daos_server.yml
*******

And yes, I am running FIO from one of the nodes.

Is there anything you see that I should modify or investigate? Thank you very much for the help!

Join daos@daos.groups.io to automatically receive all group messages.