Skip to main content
`Ctrl`+`K`

# smallpond

Smallpond is a lightweight distributed data processing framework. It uses duckdb as the compute engine and stores data in parquet format on a distributed file system (e.g. 3FS).

## Why smallpond?

- Performance: Smallpond uses DuckDB to deliver native-level performance for efficient data processing.

- Scalability: Leverages high-performance distributed file systems for intermediate storage, enabling PB-scale data handling without memory bottlenecks.

- Simplicity: No long-running services or complex dependencies, making it easy to deploy and maintain.

- Getting Started
- Internals

- API Reference
  - High-level API
    - DataFrame
  - Low-level API
    - Dataset
    - Nodes
    - Tasks
    - Execution