DeepSeek/3FS
1
0
Fork 0
mirror of https://github.com/deepseek-ai/3FS synced 2025-05-11 16:10:35 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Update deployment doc (#26)

Browse Source 
* Add commands to install dependencies

* Manually save admin token to a file
This commit is contained in:
Yiliang Xiong 2025-03-01 13:26:45 +08:00 committed by GitHub
parent fcfbb0f359
commit 4e8c49f5ec
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
2 changed files with 15 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
deploy/README.md
View File

@ -267,10 +267,11 @@ Install `storage` service on **storage** node.
```bash ```bash
/opt/3fs/bin/admin_cli -cfg /opt/3fs/etc/admin_cli.toml --config.mgmtd_client.mgmtd_server_addresses '["RDMA://192.168.1.1:8000"]' "user-add --root --admin 0 root" /opt/3fs/bin/admin_cli -cfg /opt/3fs/etc/admin_cli.toml --config.mgmtd_client.mgmtd_server_addresses '["RDMA://192.168.1.1:8000"]' "user-add --root --admin 0 root"
``` ```
Save the admin token to `/opt/3fs/etc/token.txt`. The admin token is printed to the console, save it to `/opt/3fs/etc/token.txt`.
2. Generate `admin_cli` commands to create storage targets on 5 storage nodes (16 SSD per node, 6 targets per SSD). 2. Generate `admin_cli` commands to create storage targets on 5 storage nodes (16 SSD per node, 6 targets per SSD).
- Follow instructions at [here](data_placement/README.md) to install Python packages. - Follow instructions at [here](data_placement/README.md) to install Python packages.
```bash ```bash
pip install -r ~/3fs/deploy/data_placement/requirements.txt
python ~/3fs/deploy/data_placement/src/model/data_placement.py \ python ~/3fs/deploy/data_placement/src/model/data_placement.py \
-ql -relax -type CR --num_nodes 5 --replication_factor 3 --min_targets_per_disk 6 -ql -relax -type CR --num_nodes 5 --replication_factor 3 --min_targets_per_disk 6
python ~/3fs/deploy/data_placement/src/setup/gen_chain_table.py \ python ~/3fs/deploy/data_placement/src/setup/gen_chain_table.py \

14
deploy/data_placement/README.md
View File

@ -6,9 +6,21 @@ Suppose we are going to setup a small 3FS cluster:
- 16 SSDs attached to each node - 16 SSDs attached to each node
- 6 storage targets on each SSD - 6 storage targets on each SSD
## Install dependencies
The data placement problem is formulated using [Pyomo](https://www.pyomo.org/) and solved with the [HiGHS](https://highs.dev/) solver. Install them and other dependencies:
```bash
$ cd deploy/data_placement
$ pip install -r requirements.txt
```
## Generate chain table
First generate a solution of the data placement problem. First generate a solution of the data placement problem.
```bash ```bash
$ cd deploy/data_placement
$ python src/model/data_placement.py -ql -relax -type CR --num_nodes 5 --replication_factor 3 --min_targets_per_disk 6 --init_timelimit 600 $ python src/model/data_placement.py -ql -relax -type CR --num_nodes 5 --replication_factor 3 --min_targets_per_disk 6 --init_timelimit 600
... ...
@ -45,7 +57,7 @@ $ python src/model/data_placement.py -ql -relax -type CR --num_nodes 5 --replica
Note that some combinations of `--num_nodes` and `--replication_factor` may have no solution. Note that some combinations of `--num_nodes` and `--replication_factor` may have no solution.
Then generate commands to create/remove storage targets. Then generate commands to create/remove storage targets and the chain table.
```bash ```bash
$ python src/setup/gen_chain_table.py --chain_table_type CR --node_id_begin 10001 --node_id_end 10005 --num_disks_per_node 16 --num_targets_per_disk 6 --incidence_matrix_path output/DataPlacementModel-v_5-b_10-r_6-k_3-λ_2-lb_1-ub_1/incidence_matrix.pickle $ python src/setup/gen_chain_table.py --chain_table_type CR --node_id_begin 10001 --node_id_end 10005 --num_disks_per_node 16 --num_targets_per_disk 6 --incidence_matrix_path output/DataPlacementModel-v_5-b_10-r_6-k_3-λ_2-lb_1-ub_1/incidence_matrix.pickle