DeepSeek/smallpond
1
0
Fork 0
mirror of https://github.com/deepseek-ai/smallpond synced 2025-05-31 10:57:07 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

update readme

Browse Source 
Signed-off-by: Runji Wang <runji@deepseek.com>
This commit is contained in:
Runji Wang 2025-02-27 19:45:13 +08:00
parent 770aa417d5
commit ed112db42a
1 changed files with 9 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

14
README.md
View File

@ -5,7 +5,7 @@
[![Docs](https://img.shields.io/badge/docs-latest-brightgreen.svg)](https://deepseek-ai.github.io/smallpond/)
[![License](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
A lightweight data processing framework built on DuckDB and [3FS].
A lightweight data processing framework built on [DuckDB] and [3FS].
## Features
@ -55,11 +55,15 @@ For detailed guides and API reference:
## Performance
We executed the [Gray Sort benchmark] using [smallpond] on a cluster comprising 50 compute nodes and 25 storage nodes running [3FS]. The benchmark sorted 110.5TiB of data in 30 minutes and 14 seconds, achieving a throughput of 3.66TiB/min.
We evaluated smallpond using the [GraySort benchmark] ([script]) on a cluster comprising 50 compute nodes and 25 storage nodes running [3FS]. The benchmark sorted 110.5TiB of data in 30 minutes and 14 seconds, achieving an average throughput of 3.66TiB/min.
[3FS]: https://github.com/deepseek-ai/3fs
[Gray Sort benchmark]: https://sortbenchmark.org/
[smallpond]: benchmarks/gray_sort_benchmark.py
Details can be found in [3FS - Gray Sort].
[DuckDB]: https://duckdb.org/
[3FS]: https://github.com/deepseek-ai/3FS
[GraySort benchmark]: https://sortbenchmark.org/
[script]: benchmarks/gray_sort_benchmark.py
[3FS - Gray Sort]: https://github.com/deepseek-ai/3FS?tab=readme-ov-file#2-graysort
## Development