seaweedfs: A simple and highly scalable distributed file system.
What problem it solves
Organizations need a distributed storage layer that can serve objects, files, tables, and vectors while keeping administration and horizontal expansion straightforward. SeaweedFS addresses this by providing these capabilities on shared storage and allowing capacity to grow by adding volume servers.
Key features
Unified distributed storage — A single cluster exposes S3 objects, POSIX files, lakehouse tables, and vector buckets over shared storage. The architecture separates master, volume, and filer responsibilities.
Source: README
S3-compatible object storage — The S3 gateway provides object, bucket, table, vector, IAM, and STS APIs through one endpoint. It also supports versioning, lifecycle rules, multipart uploads, presigned URLs, policies, encryption, audit logs, quotas, and rate limiting.
Source: README
Lakehouse and vector support — S3 Table Buckets store Apache Iceberg tables by default or Lance tables for vectors and multimodal data, with built-in catalog interfaces. S3 Vector Buckets keep embeddings alongside objects and tables and support vector management and nearest-vector queries.
Source: README
Horizontal capacity and metadata choices — Capacity can be expanded by starting volume servers, while stateless filer and S3 gateways can be placed behind a load balancer. Filer metadata can be stored in supported database systems, including LevelDB, RocksDB, SQLite, MySQL, PostgreSQL, Cassandra, HBase, MongoDB, Redis, and others.
Source: README
Cloud caching and tiering — Remote object storage can be mounted and cached, with metadata retained locally while content is fetched on demand. Warm volumes can also be moved to cloud storage while retaining access through the cluster.
Source: README
Quick start
Copied verbatim from the project's README.
curl -fsSL https://raw.githubusercontent.com/seaweedfs/seaweedfs/master/install.sh | bashAWS_ACCESS_KEY_ID=admin \
AWS_SECRET_ACCESS_KEY=secret \
S3_BUCKET=my-bucket \
./weed mini -dir=./dataAWS_ACCESS_KEY_ID=admin AWS_SECRET_ACCESS_KEY=secret \
aws --endpoint-url http://localhost:8333 s3 cp README.md s3://my-bucket/Who it may suit
Inference: may suit operators and developers who need a shared storage platform for S3-compatible objects, filesystem access, analytical tables, or vector workloads. Inference: may also suit teams that want to add storage capacity incrementally and choose their own filer metadata store.
Topics to discuss
Which deployment path should be documented more deeply for first-time operators: the single command, Docker, Docker Compose, Helm, or separate processes?
Which S3 compatibility scenarios should the community prioritize in integration examples and CI coverage?
How should operators choose a filer metadata store and replication strategy for their expected workload?
Written by ReadmeX AI from the project's README, source and public GitHub data; it may contain mistakes.
Read the original README https://github.com/seaweedfs/seaweedfs#readme
ReadmeX opened this community for a trending project; it is not run by the project. Maintainers can claim it from the community page.
Comments
…