Software infrastructure
Distributed software that can explain what happened.
Herder builds execution systems for workloads that must survive worker failure, restart without losing state, and show where performance is actually spent.
The problem
A queue is not enough when work is valuable. A worker may stop after finishing a job but before reporting it. A coordinator may restart while work is in flight. A late result may arrive after the assignment has changed. The software needs rules for each case, a durable record, and a way to test the result.
The work
| State | Write-ahead records, durable projections, idempotent updates, and restart behavior. |
|---|---|
| Coordination | Worker assignment, leases, retries, requeue, stale-result rejection, and dead-letter handling. |
| Measurement | Fixed workloads, regression checks, throughput tests, wait analysis, and controlled architecture experiments. |
| Handoff | Source context, test procedures, operating limits, and records that explain what was verified. |
Evidence
Herder's internal coordinator-worker runtime uses Rust, Tokio, tonic gRPC, a CRC-framed write-ahead log, and SQLite state projection. It was built as the firm's own execution and benchmarking infrastructure.
| Failure test | 100,000 jobs, 17 worker crashes, and 2 coordinator restarts. |
|---|---|
| Result | 0 unaccounted tasks after recovery. |
| Use | Measure runtime changes while preserving a deterministic correctness baseline. |
Describe the workload, the failure case, and the answer the system must retain.
Discuss a reliability problem