DeepSeek/3FS
1
0
Fork 0
mirror of https://github.com/deepseek-ai/3FS synced 2025-06-26 18:16:45 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Add commands to install dependencies

Browse Source
This commit is contained in:
Yiliang Xiong 2025-03-01 13:12:43 +08:00
parent fcfbb0f359
commit 1761c27250
1 changed files with 13 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

14
deploy/data_placement/README.md
View File

@ -6,9 +6,21 @@ Suppose we are going to setup a small 3FS cluster:
- 16 SSDs attached to each node
- 6 storage targets on each SSD
## Install dependencies
The data placement problem is formulated using [Pyomo](https://www.pyomo.org/) and solved with the [HiGHS](https://highs.dev/) solver. Install them and other dependencies:
```bash
$ cd deploy/data_placement
$ pip install -r requirements.txt
```
## Generate chain table
First generate a solution of the data placement problem.
```bash
$ cd deploy/data_placement
$ python src/model/data_placement.py -ql -relax -type CR --num_nodes 5 --replication_factor 3 --min_targets_per_disk 6 --init_timelimit 600
...
@ -45,7 +57,7 @@ $ python src/model/data_placement.py -ql -relax -type CR --num_nodes 5 --replica
Note that some combinations of `--num_nodes` and `--replication_factor` may have no solution.
Then generate commands to create/remove storage targets.
Then generate commands to create/remove storage targets and the chain table.
```bash
$ python src/setup/gen_chain_table.py --chain_table_type CR --node_id_begin 10001 --node_id_end 10005 --num_disks_per_node 16 --num_targets_per_disk 6 --incidence_matrix_path output/DataPlacementModel-v_5-b_10-r_6-k_3-λ_2-lb_1-ub_1/incidence_matrix.pickle