HIP-302: Hanzo Replicate --- Encrypted SQLite + ZapDB Durability for Base Services

Full specification: See ~/work/hanzo/papers/hip-0302-replicate.tex (LaTeX) and the compiled PDF hip-0302-replicate.pdf.

See also HIP-0107 (Streaming Replication over VFS) for the unified VFS-backed approach that supersedes the per-backend sink code referenced in this HIP. HIP-0302 stays Final as the spec for the SQLite WAL and ZapDB substrate (encryption keys, recovery objectives, sidecar pattern). HIP-0107 adds one transport pipeline on top that covers SQLite, ZapDB, blockchain state, and generic logs through the same source-adapter → age → vfs.Backend flow.

Abstract

Specifies how all Hanzo Base-powered services achieve durable state persistence through continuous encrypted streaming replication to S3-compatible storage using hanzoai/replicate and hanzoai/age. Covers two storage engines:

1. SQLite (WAL-based) — used by Base services (IAM, KMS, ATS, BD, TA, Tasks)
2. ZapDB (incremental backup) — used by high-throughput KV workloads (Gateway session cache, PubSub durable queues, Insights event buffer)

Components

Recovery Objectives

Affected Services

SQLite replication: IAM, KMS, ATS, BD, TA, Tasks, and all future Base services.

ZapDB replication: Gateway (session cache), PubSub (durable queue state), Insights (event buffer), and any service using hanzoai/kv in embedded mode.

Key Design Decisions

1. Three encryption layers: sqlcipher (disk), age (S3), TLS (transport). Independent. Compromise of one does not compromise others.
2. emptyDir replaces PVC: Pods restore from S3 on startup. No PVC scheduling constraints.
3. Per-service age keypairs: Derived from master key via HKDF with hanzo:replicate: domain separator. Cross-service access requires master key.
4. K8s sidecar pattern: Init container restores; sidecar replicates continuously.
5. ZAP binary format: ZapDB replication uses the ZAP frame format (magic 0x5A415001) for incremental backup. Frames are age-encrypted and streamed to S3 in 500ms batches.
6. Post-quantum hybrid: All age encryption uses ML-KEM-768 + X25519 hybrid mode (age v1.3.0+, age1pq key prefix). NIST FIPS 203 compliant.
7. Cloud HSM: Master keys stored in Cloud HSM (FIPS 140-2 Level 3). Key material never leaves HSM boundary.
8. ML-DSA-65 JWT: IAM issues JWT tokens signed with ML-DSA-65 (FIPS 204). JWKS validation uses PQ-safe signatures.
9. PQ TLS: X25519MLKEM768 as first curve in TLS 1.3 supported groups.

NIST Standards Adopted

Complete PQ Scorecard

All 13 cryptographic layers are deployed on devnet, testnet, and mainnet.

EOA mitigation: EOA transactions use secp256k1 ECDSA for wallet compatibility. PQ finality is achieved because Quasar consensus validators sign blocks with BLS + Corona + ML-DSA. A quantum adversary who forges an EOA signature still cannot finalize a block without compromising all three consensus assumptions.

Quasar Consensus

Quasar is a triple-hybrid consensus protocol using three independent hardness assumptions:

Block finality requires valid signatures from all three schemes. An adversary must break discrete log AND Module-LWE AND Module-SIS simultaneously.

Smart Account PQ Signing

Smart Accounts (ERC-4337 compliant) bypass the secp256k1 constraint via signature verification precompiles:

• SafeMLDSASigner: Validates ML-DSA-65 signatures via precompile at 0x0130/0x0131. Implements ERC-1271.
• SafeCoronaSigner: Validates Corona lattice signatures via precompile at 0x0150/0x0151.
• QuantumSafe: Base contract for Smart Accounts. Routes verification to the appropriate PQ precompile.

On-Chain Precompiles

All activated at genesis on all networks.

Cloud HSM for Master Keys

Master encryption keys are stored in Cloud HSM (GCP Cloud KMS, FIPS 140-2 Level 3 certified Cavium HSMs). The master key never leaves the HSM boundary. Wrap, unwrap, and sign operations execute inside the HSM. Three keyrings per ecosystem (devnet, testnet, mainnet), 12 keys total. Mainnet keys use HSM protection level.

Threshold Signing

Four threshold protocols deployed:

Harvest-Now-Decrypt-Later: Closed

The full PQ stack -- ML-KEM-768 hybrid encryption (S3 backups), X25519MLKEM768 TLS (transit), ML-DSA-65 JWT signing (auth), PQ KEM for MPC transport and key shares, Cloud HSM for master keys, and Quasar triple-hybrid consensus -- closes the HNDL attack vector. An adversary who captures data today cannot decrypt it with a future quantum computer.

Regulatory Compliance

ZapDB Streaming Replication

ZapDB is a high-throughput KV store that does not use WAL. Instead, it produces incremental backup frames in ZAP binary format:

Frame header (16 bytes):
  [0:4]   magic    = 0x5A415001 ("ZAP\x01")
  [4:8]   frame_id = monotonic uint32
  [8:12]  length   = payload length in bytes
  [12:14] checksum = CRC-16 of payload
  [14:15] flags    = 0x01=snapshot, 0x02=delta, 0x04=compressed
  [15:16] reserved = 0x00

The hanzoai/zapdb-replicator sidecar reads ZAP frames from a Unix socket, encrypts each frame with age, and streams to S3. On restore, the init container downloads the latest snapshot frame and applies delta frames in order.

S3 path convention:

s3://hanzo-replicate-{env}/{service}/{instance_id}/zapdb/
├── snapshots/{frame_id}.snap.zap.age
├── deltas/{start_frame}_{end_frame}.delta.zap.age
└── latest

Cross-Ecosystem Compatibility

This standard is adopted by LP-102 (Lux) and ZIP-0803 (Zoo) with ecosystem-specific HKDF prefixes:

References

• Litestream: https://litestream.io
• age: https://age-encryption.org
• LP-102: Lux Encrypted Streaming Replication
• ZIP-0803: Zoo Encrypted Streaming Replication
• HIP-0027: Secrets Management Standard
• HIP-0032: Object Storage Standard
• HIP-0065: Backup & Disaster Recovery Standard