Date   

Re: Questions about Daos consistency

段世博
 

I think a lease mechanism is needed, and modifications cannot be made before the server lease expires


DAOS Community Update / Oct'21

Lombardi, Johann
 

Hi there,

 

Please find below the DAOS community newsletter for October 2021.

 

Past Events (September)

  • 5th EAGE workshop on HPC for Upstream (September 6-8)
    Leveraging DAOS file system for seismic data storage.

Merna Moawad, Amr Nasr, Omar Marzouk, Mohamed AlHarouni (Brightskies)

Johann Lombardi, Mohamad Chaarawi, Philippe Thierry (Intel)

Upcoming Events

 

Release

  • We continue to make good progress towards our 2.0 release and should tag a new test build (see v1.3.106-tb) soon.
  • Major recent changes:
    • Intercept __open_2 in libioil
    • Remove “highest aggregation epoch” field which was a leftover from old concurrency control model
    • Rework VOS epoch range discard
    • Make label for pools mandatory
    • Hyphens are now supported in both pool and container labels
    • Many more erasure code fixes
    • Quiet noisy SPDK/DPDK messages introduced with the upgrade to v21.07
    • Return most recent snapshot in container query
    • Integrate CART self-test with the DAOS agent (no need to dump attach info file any longer)
    • Fix a few memory leaks on error paths
    • Add “enable_hotplug” field to the yaml file to enable SSD hotplug support
    • Fix spurious pool handle eviction from the agent under memory pressure
    • Add several swim improvements to support “high-latency” network
    • Fix race with double RPC completions
    • Bump RDMA threshold for I/Os from 3.5KiB to 19KiB
    • Leverage helper xstreams (when available) for DTX batched commit
  • What is coming:
    • Fixes, fixes and more fixes.
    • Libfabric upgrade to grab some critical rxm/verbs fixes for DAOS.
    • Investigate an issue with NVMe unmap (aka trim) after repeated reformats.

 

R&D

  • Major features under development:
    • Checksum scrubbing
    • LDMS plugin to export DAOS metrics
    • API to collect libdaos metrics to be integrated with Darshan
    • Multi-user dfuse
    • More aggressive caching in dfuse for AI APPs
    • Design for catastrophic recovery / fsck
  • Pathfinding:
    • MariaDB DAOS engine with predicate pushdown to the DAOS storage nodes
    • Leveraging the Intel Data Streaming Accelerator (DSA) to accelerate DAOS

 

News

  • New look for https://docs.daos.io (dark theme)
  • Staffing:
    • We welcome Michael Hennecke to the Intel DAOS team.

 

---------------------------------------------------------------------
Intel Corporation SAS (French simplified joint stock company)
Registered headquarters: "Les Montalets"- 2, rue de Paris,
92196 Meudon Cedex, France
Registration Number:  302 456 199 R.C.S. NANTERRE
Capital: 4,572,000 Euros

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.


[DUG'21] Save the date & call for presentations!

Lombardi, Johann
 

Dear DAOS community,

 

We are pleased to announce the 5th annual DAOS User Group (DUG) around SC’21. The DUG will be again virtual this year with live presentations on Nov 19 from 9am to 1pm (Central Time). Details about the conference (e.g. dial-in info, agenda) will be updated here once finalized.

 

As previous years, we would like to invite community members to submit presentation proposals (i.e. title + short summary) to daos-info@daos.groups.io

We encourage any feedback and would like to hear from you on your experience with DAOS, future plans, what you have contributed or intend to contribute, what worked … and did not work so well. We are looking forward to your submissions!

 

Take care.

Johann – on behalf of the Intel DAOS Team

---------------------------------------------------------------------
Intel Corporation SAS (French simplified joint stock company)
Registered headquarters: "Les Montalets"- 2, rue de Paris,
92196 Meudon Cedex, France
Registration Number:  302 456 199 R.C.S. NANTERRE
Capital: 4,572,000 Euros

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.


Re: Looking for a really simple guide

Durfey, Craig
 

Can you provide more details regarding this issue? Particularly:

  • What type of system are you are trying to enable?
  • Did you start from or use the CentOS setup guide?
  • If you used the CentOS Guide, what pieces of documentation would have been helpful to have contained in that Guide?

- Thanks in advance


Looking for a really simple guide

adam@...
 

Hi, for the past few days I've been struggling to a test instance of DAOS running.
I am literally trying to just spin up the smallest possible example of a system to examine the functionality in practice.
I have tried building from packages, the docker container, from source, all on CentOS 7, but have failed at various points.
My question is therefore, is there an easy to consume "for idiots" guide. I'm not new to this but the existing documentation is spread scattershot across various sections of the Administration and User Guides and has me going back and forth to get this implemented.
Is there a hidden gem of guidance out there that I've missed, on spinning up just a single system, without the need for SCM or NVMe, simply to demo the technology?


Re: Questions about Daos consistency

段世博
 

If a DTX starts after this copy has a network partition, this replica will not have a record of this DTX. If the client reads the data modified by this DTX after this DTX is committed, it will get outdated data on this replica,because it is not will re-read to Leader.


Re: Fio with I/O Engine=DFS

shmatsuu@...
 

Hi Johann,
 
Thank you very much for investigating this issue! I was able to solve the issue a few days ago, and I apologize for not updating my status earlier.
To me, the root cause of the issue seems to be the version inconsistency between DAOS server and DASO client header files (daos.h, daos_fs.h, daos_pool.h) that I have used to compile the fio dfs engine. The DAOS server was v.1.2, but the client header files were not. It seems that the fio was trying to connect daos pools with a different version of APIs and failed to connect. After replacing the client headers files with those v.1.2 and recompiling the fio dfs engine, I was able to run the fio dfs engine successfully. Again, thank you very much for your help, and my sincere apologies for not letting you know my status earlier.

Thank you very much,
---
Shohei


Re: Questions about Daos consistency

Yong, Fan
 

The read request can be sent to a non-leader replica. If the server for such replica is out of the cluster (because of network partition) but itself does not know yet, then related DTX on such replica may miss the (async) DTX commit RPC as to still be in ‘prepared’ status when the read RPC arrived. Under such case, if the client also uses old pool map, then the client and the server will not be aware of the stale pool map. But ‘prepared’ DTX on non-leader replica is uncertain for the read request, the non-leader will return -DER_INPROGRESS (-2018) to the client that will notify the client to re-read from leader replica. We assume that the leader replica is healthy at that time. When the client re-read from the leader replica, it will know its pool map is stale, then refresh its pool map, and then get the latest data from the leader replica.

 

--

Regards,

Nasf

 

From: <daos@daos.groups.io> on behalf of 段世博 <duanshibo.d@...>
Reply-To: "daos@daos.groups.io" <daos@daos.groups.io>
Date: Thursday, September 23, 2021 at 13:09
To: "daos@daos.groups.io" <daos@daos.groups.io>
Subject: [daos] Questions about Daos consistency

 

DAOS uses two-phase commit to ensure consistency between replicas. According to "src/vos/readme.md", read requests can be sent to any replica, but if this replica server has a network partitionthe client uses the old Pool map to access the old replica server, then the latest committed data cannot be seen. How does DAOS avoid this situation?


Questions about Daos consistency

段世博
 

DAOS uses two-phase commit to ensure consistency between replicas. According to "src/vos/readme.md", read requests can be sent to any replica, but if this replica server has a network partition(the client uses the old Pool map to access the old replica server), then the latest committed data cannot be seen. How does DAOS avoid this situation?


Re: Fio with I/O Engine=DFS

Lombardi, Johann
 

Hi Shohei,

 

I have just tried fio-3.19 with the dfs engine against both DAOS 1.2 (using UUIDS) and master (using UUIDs and labels) and couldn’t reproduce.

Could you please check the pool UUID with dmg pool list and also send me the output of dmg pool query --pool $PUUID?

 

Cheers,

Johann

 

From: <daos@daos.groups.io> on behalf of "Lombardi, Johann" <johann.lombardi@...>
Reply-To: "daos@daos.groups.io" <daos@daos.groups.io>
Date: Thursday 9 September 2021 at 07:57
To: "daos@daos.groups.io" <daos@daos.groups.io>
Subject: Re: [daos] Fio with I/O Engine=DFS

 

Hm, I wonder whether I broke something with my latest fio upstream patch.

Could you please try to revert commit 2819492b94f9ef901178e6972e0e5784ef3a8be9 locally, rebuild fio and retry?

Thanks in advance.

 

Cheers,

Johann

 

From: <daos@daos.groups.io> on behalf of "shmatsuu@..." <shmatsuu@...>
Reply-To: "daos@daos.groups.io" <daos@daos.groups.io>
Date: Thursday 9 September 2021 at 01:38
To: "daos@daos.groups.io" <daos@daos.groups.io>
Subject: Re: [daos] Fio with I/O Engine=DFS

 

Hi Johann,

Thank you. Here is an example how I set POOL and CONT env vars to. Yes, I'm using DAOS 1.2 and fio version fio-3.27-84-gf7942.

[Setting Env Vars]
 export POOL=4a6fa2f6-f2aa-40d5-9092-63ea46641887
 export CONT=1ed2c63e-9c20-4746-9ec4-03ce57c5d5a2 

[FIO Version]
fio --version
fio-3.27-84-gf7942

Thank you very much,
---
Shohei

 

---------------------------------------------------------------------
Intel Corporation SAS (French simplified joint stock company)
Registered headquarters: "Les Montalets"- 2, rue de Paris,
92196 Meudon Cedex, France
Registration Number:  302 456 199 R.C.S. NANTERRE
Capital: 4,572,000 Euros

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.


Re: Fio with I/O Engine=DFS

Lombardi, Johann
 

Hm, I wonder whether I broke something with my latest fio upstream patch.

Could you please try to revert commit 2819492b94f9ef901178e6972e0e5784ef3a8be9 locally, rebuild fio and retry?

Thanks in advance.

 

Cheers,

Johann

 

From: <daos@daos.groups.io> on behalf of "shmatsuu@..." <shmatsuu@...>
Reply-To: "daos@daos.groups.io" <daos@daos.groups.io>
Date: Thursday 9 September 2021 at 01:38
To: "daos@daos.groups.io" <daos@daos.groups.io>
Subject: Re: [daos] Fio with I/O Engine=DFS

 

Hi Johann,

Thank you. Here is an example how I set POOL and CONT env vars to. Yes, I'm using DAOS 1.2 and fio version fio-3.27-84-gf7942.

[Setting Env Vars]
 export POOL=4a6fa2f6-f2aa-40d5-9092-63ea46641887
 export CONT=1ed2c63e-9c20-4746-9ec4-03ce57c5d5a2 

[FIO Version]
fio --version
fio-3.27-84-gf7942

Thank you very much,
---
Shohei

---------------------------------------------------------------------
Intel Corporation SAS (French simplified joint stock company)
Registered headquarters: "Les Montalets"- 2, rue de Paris,
92196 Meudon Cedex, France
Registration Number:  302 456 199 R.C.S. NANTERRE
Capital: 4,572,000 Euros

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.


Re: Fio with I/O Engine=DFS

shmatsuu@...
 

Hi Johann,

Thank you. Here is an example how I set POOL and CONT env vars to. Yes, I'm using DAOS 1.2 and fio version fio-3.27-84-gf7942.

[Setting Env Vars]
 export POOL=4a6fa2f6-f2aa-40d5-9092-63ea46641887
 export CONT=1ed2c63e-9c20-4746-9ec4-03ce57c5d5a2 

[FIO Version]
fio --version
fio-3.27-84-gf7942

Thank you very much,
---
Shohei


Re: Fio with I/O Engine=DFS

Lombardi, Johann
 

Hi Shohei,

 

Could you please give us an example of what you set the POOL and CONT env vars to? Also, I assume that you are using DAOS 1.2, right?

 

Cheers,

Johann

 

From: <daos@daos.groups.io> on behalf of "shmatsuu@..." <shmatsuu@...>
Reply-To: "daos@daos.groups.io" <daos@daos.groups.io>
Date: Wednesday 8 September 2021 at 16:30
To: "daos@daos.groups.io" <daos@daos.groups.io>
Subject: [daos] Fio with I/O Engine=DFS

 

Hi, 

I have a question regarding fio with dfs io engine.
I'm trying to run fio benchmarks with ioengine=dfs, by following the example here (https://docs.daos.io/admin/performance_tuning/), but getting the below error message. I'm sure that my Pool UUID and Container UUID are correct.
Can anyone point out for me what I'm doing wrong or any possible cause for the error, please? Or can you please share with me how you successfully run fio with ioengine=dfs. 

[How I run the fio with ioengine=dfs]
(1) dmg pool create --size=<specific size> --user=<fio running user>@ --group=<fio running user group>@
(2) daos cont create --pool=<Pool UUID>  --user=<fio running user>@ --group=<fio running user group>@ --type=POSIX
(3) export POOL=<Pool UUID>
(4) export CONT=<Container UUID>
(5) fio ./examples/dfs.fio

[Error Messages]

daos-seqwrite: (g=0): rw=write, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=dfs, iodepth=1

daos-seqread: (g=1): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=dfs, iodepth=1

daos-randwrite: (g=2): rw=randwrite, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=dfs, iodepth=1

daos-randread: (g=3): rw=randread, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=dfs, iodepth=1

fio-3.27-84-gf7942

Starting 4 processes

mgmt ERR  src/mgmt/cli_mgmt.c:815 dc_mgmt_get_pool_svc_ranks() 35396364: MGMT_POOL_GET_SVCRANKS rpc failed to all 1 ranks, DER_NONEXIST(-1005): 'The specified entity does not exist'

pool ERR  src/pool/cli.c:195 dc_pool_choose_svc_rank() 35396364: dc_mgmt_get_pool_svc_ranks() failed, DER_NONEXIST(-1005): 'The specified entity does not exist'

pool ERR  src/pool/cli.c:512 dc_pool_connect() 35396364: cannot find pool service: DER_NONEXIST(-1005): 'The specified entity does not exist'

Failed to connect to pool -1005

fio: io engine dfs init failed.

Thank you very much, 
---
Shohei

---------------------------------------------------------------------
Intel Corporation SAS (French simplified joint stock company)
Registered headquarters: "Les Montalets"- 2, rue de Paris,
92196 Meudon Cedex, France
Registration Number:  302 456 199 R.C.S. NANTERRE
Capital: 4,572,000 Euros

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.


Fio with I/O Engine=DFS

shmatsuu@...
 

Hi, 

I have a question regarding fio with dfs io engine.
I'm trying to run fio benchmarks with ioengine=dfs, by following the example here (https://docs.daos.io/admin/performance_tuning/), but getting the below error message. I'm sure that my Pool UUID and Container UUID are correct.
Can anyone point out for me what I'm doing wrong or any possible cause for the error, please? Or can you please share with me how you successfully run fio with ioengine=dfs. 

[How I run the fio with ioengine=dfs]
(1) dmg pool create --size=<specific size> --user=<fio running user>@ --group=<fio running user group>@
(2) daos cont create --pool=<Pool UUID>  --user=<fio running user>@ --group=<fio running user group>@ --type=POSIX
(3) export POOL=<Pool UUID>
(4) export CONT=<Container UUID>
(5) fio ./examples/dfs.fio

[Error Messages]
daos-seqwrite: (g=0): rw=write, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=dfs, iodepth=1
daos-seqread: (g=1): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=dfs, iodepth=1
daos-randwrite: (g=2): rw=randwrite, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=dfs, iodepth=1
daos-randread: (g=3): rw=randread, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=dfs, iodepth=1
fio-3.27-84-gf7942
Starting 4 processes
mgmt ERR  src/mgmt/cli_mgmt.c:815 dc_mgmt_get_pool_svc_ranks() 35396364: MGMT_POOL_GET_SVCRANKS rpc failed to all 1 ranks, DER_NONEXIST(-1005): 'The specified entity does not exist'
pool ERR  src/pool/cli.c:195 dc_pool_choose_svc_rank() 35396364: dc_mgmt_get_pool_svc_ranks() failed, DER_NONEXIST(-1005): 'The specified entity does not exist'
pool ERR  src/pool/cli.c:512 dc_pool_connect() 35396364: cannot find pool service: DER_NONEXIST(-1005): 'The specified entity does not exist'
Failed to connect to pool -1005
fio: io engine dfs init failed.

Thank you very much, 
---
Shohei


DAOS Community Update / Sep'21

Lombardi, Johann
 

Hi there,

 

Please find below the DAOS community newsletter for September 2021.

 

Upcoming Events

  • 5th EAGE workshop on HPC for Upstream (September 6-8)

Leveraging DAOS file system for seismic data storage.

Merna Moawad, Amr Nasr, Omar Marzouk, Mohamed AlHarouni (Brightskies)

Johann Lombardi, Mohamad Chaarawi, Philippe Thierry (Intel)

  • 19th workshop on HPC in meteorology (September 20-24) with two DAOS presentations:
    • First Experiences with CDI-PIO on DAOS

Michael HENNECKEChristoph POSPIECH (Lenovo), Sigrun EGGERLING (Lenovo)
Thomas Mr JAHNS, Panagiotis ADAMIDIS (DKRZ)

    • Accelerating Storage with Optane & DAOS
      Johann Lombardi (Intel)

Nicolau Manubens (ECMWF)

  • LAD’21 (September 28-30)
    DAOS/Lustre Integration

Kevin Harms (ALCF)
Johann Lombardi, Bruno Faccini (Intel)

  • DAOS User Group (DUG’21) at SC'21 - details coming soon!

 

Release

  • We continue to make good progress towards our 2.0 release with a new test build (see v1.3.105-tb) tagged last week.
  • Major recent changes:
    • dmg & daos utility uniformization along with many usability improvements
    • supports for pool and container labels (no need to use UUID any longer!)
    • the DAOS engine is now maintaining a list of a pre-registered buffers for RDMA to boost read/write performance.
    • dfuse now officially supports writeback caching.
    • Interception library now supports an environment variable to log interception activity.
    • Many erasure code fixes
    • Engine metrics can be exported via Prometheus.
    • SPDK upgrade to v21.07
    • Mercury upgrade to v2.0.1
  • What is coming:
    • Fixes, fixes and more fixes.

 

R&D

  • Major features under development:
    • Checksum scrubbing
    • LDMS plugin to export DAOS metrics
    • API to collect libdaos metrics to be integrated with Darshan
    • Multi-user dfuse
  • Pathfinding:
    • MariaDB DAOS engine with predicate pushdown to the DAOS storage nodes.
    • Leveraging the Intel Data Streaming Accelerator (DSA) to accelerate DAOS

 

News

---------------------------------------------------------------------
Intel Corporation SAS (French simplified joint stock company)
Registered headquarters: "Les Montalets"- 2, rue de Paris,
92196 Meudon Cedex, France
Registration Number:  302 456 199 R.C.S. NANTERRE
Capital: 4,572,000 Euros

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.


Re: protential deadlock problem in daos/src/client/dfuse/il/int_posix.c

Lombardi, Johann
 

Hi Chengfeng,

 

Good catch and thanks for the patch!

 

Cheers,

Johann

 

From: <daos@daos.groups.io> on behalf of "cyeaa@..." <cyeaa@...>
Reply-To: "daos@daos.groups.io" <daos@daos.groups.io>
Date: Wednesday 25 August 2021 at 15:49
To: "daos@daos.groups.io" <daos@daos.groups.io>
Subject: [daos] protential deadlock problem in daos/src/client/dfuse/il/int_posix.c

 

Hi developers,

I am about to report a protential bug, but I couldn't get access to the issue tracker in this site https://daosio.atlassian.net . I just put the description here, and thanks a lot if someone could reply.

The problem here is that the lock ioil_iog.iog_lock will become an unreleased lock if the program return at line 719. Normally this lock will be released at the end of the function, but the code in this branch(line 719) seems forget to release the lock. If this lock is required again in other place, there will be a deadlock problem. 
The link of the code is here https://github.com/daos-stack/daos/blob/1cd2ae176878393051df10c73e6adc62668ec330/src/client/dfuse/il/int_posix.c#L710



I think the fixing of it should be simple, just need to release the lock before the return statement at line 719.

Thanks,
Chengfeng

---------------------------------------------------------------------
Intel Corporation SAS (French simplified joint stock company)
Registered headquarters: "Les Montalets"- 2, rue de Paris,
92196 Meudon Cedex, France
Registration Number:  302 456 199 R.C.S. NANTERRE
Capital: 4,572,000 Euros

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.


protential deadlock problem in daos/src/client/dfuse/il/int_posix.c

cyeaa@...
 

Hi developers,

I am about to report a protential bug, but I couldn't get access to the issue tracker in this site https://daosio.atlassian.net . I just put the description here, and thanks a lot if someone could reply.

The problem here is that the lock ioil_iog.iog_lock will become an unreleased lock if the program return at line 719. Normally this lock will be released at the end of the function, but the code in this branch(line 719) seems forget to release the lock. If this lock is required again in other place, there will be a deadlock problem. 
The link of the code is here https://github.com/daos-stack/daos/blob/1cd2ae176878393051df10c73e6adc62668ec330/src/client/dfuse/il/int_posix.c#L710



I think the fixing of it should be simple, just need to release the lock before the return statement at line 719.

Thanks,
Chengfeng


Re: Question about Pool Size Expansion

shmatsuu@...
 

Hi Kevin,

Thank you very much for your response! I'll test it on my environment. 

Thank you,
---
Shohei


Re: Question about Pool Size Expansion

Harms, Kevin
 

Shohei,

my understanding is that the current implementation assumes you have servers with the same configuration, so if for example, rank0 and rank1 have 5 targets each with 10GB, when you add rank2, you will have 150GB or an additional 5 targets with 10GB each. But I have not tested it myself.

kevin

________________________________________
From: daos@daos.groups.io <daos@daos.groups.io> on behalf of shmatsuu@yahoo-corp.jp <shmatsuu@yahoo-corp.jp>
Sent: Monday, August 23, 2021 9:34 AM
To: daos@daos.groups.io
Subject: Re: [daos] Question about Pool Size Expansion

Hi Kevin,

Thanks for your reply. I noted. Maybe I'm missing something here, but does this lead us to expanding the size of an existing pool?
For example, let's say we have a pool with uuid=puuid and 100GB in size distributed over rank 0 and rank 1 now.
And, if we add a new server with rank2 with the following command, does it result in increasing the size of the pool with uuid=puuid to be more than 100GB?
To me, the following just results in distributing the data in the pool more horizontally, rank0-rank2?
dmg pool extend --pool=${puuid} --ranks=${rank2}
https://docs.daos.io/admin/pool_operations/#pool-extension

Thank you,
---
Shohei


Re: Question about Pool Size Expansion

shmatsuu@...
 

Hi Kevin,

Thanks for your reply. I noted. Maybe I'm missing something here, but does this lead us to expanding the size of an existing pool?
For example, let's say we have a pool with uuid=puuid  and 100GB in size distributed over rank 0 and rank 1 now. 
And, if we add a new server with rank2 with the following command, does it result in increasing the size of the pool with uuid=puuid to be more than 100GB? 
To me, the following just results in distributing the data in the pool more horizontally, rank0-rank2? 
dmg pool extend --pool=${puuid} --ranks=${rank2}
https://docs.daos.io/admin/pool_operations/#pool-extension

Thank you,
---
Shohei

41 - 60 of 1488