Scaling OpenClaw Fleets to Thousands of Agents

Scaling Overview

Each agent maintains a persistent connection to the network for discovery, keepalive, and event delivery. This guide describes patterns for fleets that may grow into the thousands; it is not a published capacity guarantee. Benchmark the registry, relay usage, connection fan-out, and event volume against your own deployment before setting production limits.

Per-Agent Resource Budget

Each OpenClaw host runs a Pilot daemon. Its resource footprint depends on peer fan-out, payload volume, buffering, relay use, logging, and the services enabled. Establish a budget from measurements on the host type you plan to operate:

Do not extrapolate a single-agent sample into a fleet guarantee. Run the expected number of identities, peer relationships, and services on the intended machine class, then include recovery bursts and the agent workloads themselves in the capacity margin.

Systemd Deployment Pattern

For persistent deployments, run daemons as systemd services rather than foreground processes:

# /etc/systemd/system/pilot-daemon.service
[Unit]
Description=Pilot Protocol Daemon
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
User=openclaw
Group=openclaw
ExecStart=/usr/local/bin/pilot-daemon
Restart=always
RestartSec=5
Environment=HOME=/home/openclaw

[Install]
WantedBy=multi-user.target

sudo systemctl enable pilot-daemon
sudo systemctl start pilot-daemon

For deploying fleets of agents on the same server, use templated units:

# /etc/systemd/system/pilot-agent@.service
[Unit]
Description=Pilot Agent %i
After=pilot-daemon.service
Requires=pilot-daemon.service

[Service]
Type=simple
User=openclaw
Environment=AGENT_ID=%i
ExecStart=/usr/local/bin/openclaw-worker --id %i
Restart=always
RestartSec=10

[Install]
WantedBy=multi-user.target

# Deploy 50 agents
for i in $(seq 1 50); do
  sudo systemctl enable pilot-agent@$i
  sudo systemctl start pilot-agent@$i
done

Monitoring at Scale

With hundreds of agents, you need visibility. Pilot Protocol's built-in event stream enables lightweight monitoring without external tools:

# Monitor agent on a dedicated monitoring node
pilotctl subscribe  "fleet.*"

# Each agent publishes periodic health events
pilotctl publish  "fleet.health" \
  --data '{"agent":"worker-42","status":"healthy","tasks_completed":127,"uptime_hours":48}'

For large fleets, aggregate health events on a monitoring agent that tracks:

• Agent count: How many agents are actively publishing health events
• Task throughput: Total tasks completed per minute across the fleet
• Error rate: Percentage of task failures
• Trust graph changes: New connections or revocations

Use pilotctl peers for a real-time view of your currently connected peers (encrypted/relay/direct breakdown); for network-wide public-agent discovery, query the list-agents directory. For private fleets, the monitoring agent can stream metrics directly to your existing monitoring stack.

Bottlenecks at Scale

Fleet bottlenecks depend on topology and workload, but the first places to test are predictable:

Startup and Reconnect Bursts

Many agents registering, resolving names, or rebuilding paths at once can create a thundering herd. Add jitter, bound retries, and test a coordinated restart before production.

Discovery Result Size

Broad tag searches become more expensive as the directory and result set grow. Use specific tags, pagination, caching, and staggered refresh intervals.

Relay and Event Fan-Out

Relay bandwidth and high-fan-out event streams can concentrate load. Measure direct-versus-relayed paths and size coordination, relay, and monitoring components independently.

Fleet Deployment Patterns

Three common fleet patterns are useful starting points:

Hub-and-spoke. One orchestrator agent with many worker agents. The orchestrator discovers workers by tag, submits tasks, collects results. Workers are stateless and interchangeable. This is the most common pattern for batch processing jobs.

Mesh. Every agent may connect to every other agent. This can suit small teams, but pairwise relationships grow as O(n²) and become difficult to govern as the fleet expands.

Hierarchical. Orchestrator agents manage bounded worker groups, and higher-level coordinators operate across those groups. Size each tier from measured task, event, and recovery load; do not assume organizational hierarchy automatically grants network trust.