Skip to main content

System Constraints & Considerations

Storage limits, cost considerations, memory requirements, and other constraints to consider when building on Walrus.

Consider the following practical limits and constraints when you design applications on Walrus. For current values, run walrus info.

What is the maximum blob size?​

Walrus supports blobs up to approximately 13.6GB. Check the current limit with walrus info under Maximum blob size.

When you use quilts for batch storage, each individual blob within the quilt has a limit of approximately 4GB. The quilt's internal header format imposes this per-blob limit. Check the current limit with walrus info under Maximum blob size in quilt. If you need to store data larger than 4GB, store it as a regular blob instead of within a quilt.

If your data exceeds this limit, learn more about best practices for large data uploads.

How much memory do encoding and decoding need?​

Encoding and decoding blobs requires significant memory:

  • Encoding (upload): Requires approximately 2-3x the blob size in available RAM.

  • Decoding (retrieval): Requires approximately 1.5-2x the blob size in available RAM.

For example, encoding a 5GB blob requires 10 to 15GB of available RAM.

If the system runs out of memory during encoding, the operation fails with an error such as Out of memory or Failed to allocate buffer. Split large blobs into smaller chunks to reduce memory requirements, or run the CLI on a machine with more RAM.

Aggregator operators should provision aggregators with enough memory to handle the largest blobs they expect to reconstruct.

How to handle rate limits​

Rate limits apply to both storage nodes and Sui RPC endpoints.

Storage node rate limits​

Storage nodes might rate-limit requests to prevent abuse. If you encounter HTTP 429 or Too many requests errors, implement exponential backoff in your retry logic. Using publishers and aggregators avoids direct interaction with storage nodes and handles rate limiting internally.

Sui RPC rate limits​

Public Sui RPC endpoints have request quotas. If you encounter RPC rate limit exceeded errors, configure multiple RPC endpoints for failover, cache blockchain queries where possible, or use a paid RPC service for production workloads.

How much network bandwidth do uploads use?​

Upload and download times are proportional to blob size and network speed. Walrus encodes data with erasure coding, which expands the data approximately 4.5x. As a result, uploading a 1GB blob transmits roughly 4.5GB of encoded data across the network.

To reduce upload latency:

  • Compress data before uploading if your use case allows it. Compression reduces the stored size, which reduces both storage cost and upload time.

For downloads, aggregators cache frequently accessed blobs. If your application serves the same blobs to many users, place an aggregator with caching closer to your users to reduce latency.

If you are using a machine with limited resources, consider using a publisher to offload encoding. The client uploads the raw blob once, and the publisher handles encoding and sliver distribution.

How do epoch transitions affect retrieval?​

Storage node shard assignments change every epoch. During transitions, nodes migrate data to new shard owners. This process is automatic but can briefly affect retrieval latency. In rare cases, some slivers might be temporarily unavailable until migration completes.

Applications should implement retry logic to handle transient failures during epoch transitions.

What fault tolerance does Walrus assume?​

Walrus guarantees hold as long as honest storage nodes manage more than 2/3 of storage shards (by stake weight). The system tolerates faulty or malicious nodes controlling up to 1/3 of shards.

This assumption applies both within individual storage epochs and across epoch transitions. Under normal operation, the Sui staking mechanism and economic incentives maintain this property.

For data with extreme durability requirements, consider maintaining additional off-Walrus backups.

Are public publishers and aggregators reliable?​

Public publishers and aggregators do not have formal availability guarantees. They might go offline, experience performance issues, or enforce rate limits.

For production applications:

  • Do not rely on a single publisher or aggregator.
  • Run your own publisher and aggregator infrastructure.
  • Implement failover across multiple endpoints.
  • Maintain the ability to fall back to direct CLI or SDK usage.