HIP-0120: ZAP-Native Transport & gRPC Elimination

Abstract

Hanzo has exactly one inter-service transport: ZAP, the Hanzo-native binary wire (github.com/zap-proto/*). This HIP makes the corollary a platform-wide standard:

A Hanzo service speaks ZAP, HTTP, or WS. It never speaks gRPC.

The three permitted protocols are complete for every wire Hanzo needs — ZAP for inter-service and inter-VM traffic (HIP-0106, HIP-0114), HTTP for the external request/response edge and HTTP-semantics-over-ZAP internally, and WS for long-lived bidirectional client streams. gRPC is a fourth transport with its own HTTP/2 framing, its own code-generated stub layer, and its own mutable-message data model; admitting it would violate "one and only one way to do everything" and drag google.golang.org/grpc — a large dependency and attack surface — into the graph.

This HIP specifies: the rule and its RFC 2119 force; the three permitted protocols; OTLP-over-ZAP (the shipped exemplar — OpenTelemetry spans ride the ZAP wire to the o11y collector, standard OTLP protobuf payload, ZAP transport, never OTLP-HTTP :4318 or OTLP-gRPC :4317); the pb2zap source-migration path; the design decision that there is no transparent runtime gRPC-over-ZAP shim; and the canonical conformance gate — zero first-party google.golang.org/grpc imports, enforced by grep over .go source, not by go.mod.

The standard is already shipped across four repositories — cloud, gateway, visor, ai — in four commits (see Reference Implementation). First-party gRPC imports across all four are zero. What remains is never-dialed transitive library code, honestly accounted for in Residual & Conformance.

Motivation

One transport, not two

HIP-0106 fixed ZAP as the inter-subsystem contract of the unified cloud binary: every subsystem ships a .zap schema, zapc generates the bindings, and calls resolve to direct Go method dispatch when co-resident or ZAP RPC when split. HIP-0114 fixed ZAP as the inter-VM cognitive transport for Thinking Chains. Both rest on the same premise: ZAP is the one Hanzo wire. A second RPC transport — gRPC — is not a convenience, it is a fork in the road, and "if there are two ways to do something, one is wrong." gRPC is the wrong one: Hanzo already has a native, versioned, schema-driven transport that spans in-process, node-local, and cross-node hops with one set of generated types.

ZAP is zero-copy; gRPC's message model is not

ZAP splits a message into three values with distinct lifetimes: an Input (built once, write-only), a []byte (the transport frame), and a View (read many times, zero-copy over the frame's backing bytes). Protobuf/gRPC collapses these into a single mutable message struct that is built, read, and mutated in place. The mutable model forces allocation and copy at every boundary and is precisely the model ZAP rejects. Bolting gRPC onto a ZAP stack re-imports the cost ZAP was built to remove.

The edge is JWT→ZAP; gRPC would need a parallel identity path

Per HIP-0112, external traffic terminates at ingress → gateway: TLS ends at ingress, the gateway validates the IAM JWT, strips client-supplied identity headers, and mints trusted ones. Every hop after the gateway rides ZAP with the minted identity already attached. A gRPC service would need its own HTTP/2 metadata-based auth plumbing and its own interceptor chain kept in lockstep with the gateway's header contract — a second identity path to drift out of sync. One transport means one identity path.

Supply-chain surface

google.golang.org/grpc pulls an HTTP/2 framing stack, connection/balancer machinery, and the protoc-gen-go-grpc stub layer; the OpenTelemetry gRPC exporters (otlptracegrpc, otlpmetricgrpc) and otelgrpc pull it further into telemetry. Each import is transitive CVE exposure and dependency weight for a wire Hanzo does not use. Eliminating first-party gRPC shrinks the graph to what Hanzo actually dials.

Specification

The keywords MUST, MUST NOT, SHALL, SHOULD, MAY are used per RFC 2119.

1. The rule

• A Hanzo service MUST NOT import google.golang.org/grpc (or a generated *_grpc.pb.go stub) in first-party source.
• A Hanzo service MUST NOT expose a gRPC server or dial a gRPC client for traffic between Hanzo services.
• A Hanzo service MUST carry inter-service and inter-VM traffic over ZAP, and MAY additionally terminate HTTP and WS at the external edge (§2).
• Where an external dependency is only reachable over a gRPC SDK, the service MUST prefer the provider's REST/HTTP surface and hand-roll a net/http client when a maintained REST client is unavailable (§Rationale; the visor/ai GCP compute adapters are the reference). This keeps the gRPC transport out of Hanzo's first-party graph.

2. The three permitted protocols

The rest of the ZAP family (zap-proto/zapd router, zap-proto/zap-spec) are part of the same ecosystem. Note the namespace: the ZAP transport and framework live under github.com/zap-proto/*; this is distinct from hanzoai/zap, the Hanzo SQL/KV/Datastore sidecar, which keeps the hanzoai namespace and is not a transport.

gRPC appears in none of these rows. There is no fourth row.

3. OTLP-over-ZAP (shipped exemplar)

Telemetry is the canonical demonstration that a protocol conventionally bound to gRPC can ride ZAP with no semantic change. OpenTelemetry ships an otlptrace.Client interface whose default wires are OTLP-over-gRPC (:4317) and OTLP-over-HTTP (:4318). Hanzo implements a third client whose wire is ZAP.

cloud/zaptrace/zaptrace.go is an otlptrace.Client (compile-time asserted: var _ otlptrace.Client = (*Client)(nil)) whose transport is github.com/zap-proto/http:

• Payload is unchanged. Spans are marshaled as standard OTLP protobuf. Only the transport differs — the collector receives a byte-identical ExportTraceServiceRequest.
• Wire is ZAP. UploadTraces builds a fasthttp POST http://zap/v1/traces with Content-Type: application/x-protobuf and sends it through a pooled zaphttp.Transport. The collector's ZAP receiver (zapreceiver) terminates the frame at :4319 and writes to hanzoai/datastore. Spans travel OTLP-over-ZAP — never OTLP-HTTP :4318, never OTLP-gRPC :4317.
• The envelope is hand-encoded to stay gRPC-free. The request is a single repeated ResourceSpans resource_spans = 1, so UploadTraces appends each ResourceSpans under field 1 with protowire directly from the grpc-free go.opentelemetry.io/proto/otlp/trace/v1 messages. This deliberately avoids the generated collector/trace/v1.ExportTraceServiceRequest type, whose sibling trace_service_grpc.pb.go (shipped with no build tag) would drag google.golang.org/grpc back into the module graph. The hand-encoding is byte-identical to the generated marshaler — proven by zaptrace_test.go, which stands up a real zaphttp.Server and decodes the received body with the canonical collector/trace/v1 type.
• Selection is by environment; no code branches per deploy. cloud/cmd/cloud/telemetry.go installs exactly one tracer provider:
  • OTEL_EXPORTER_ZAP_ENDPOINT set (or unset with a ZAP default of localhost:4319) → spans ride ZAP.
  • Only a legacy OTEL_EXPORTER_OTLP_* endpoint set → spans ride OTLP-HTTP (the interop/loopback fallback used while the standalone collector is folded in). This fallback is HTTP, never gRPC.
• Composition-root ownership. After installing the ZAP provider, cloud unsets OTEL_EXPORTER_OTLP_TRACES_ENDPOINT / OTEL_EXPORTER_OTLP_ENDPOINT so an embedded subsystem (notably hanzoai/ai's object.InitTelemetry) cannot install a second, competing OTLP provider that would strand later spans on :4318. Exactly one provider, one wire, deterministically.

Scope note (honest): traces ride ZAP today. A ZAP-native metrics/logs exporter is roadmap (§Residual & Conformance); the OTLP metrics/logs exporters are the current transitive-grpc source, never a dialed gRPC channel.

4. The pb2zap migration path

Porting a protobuf/gRPC call site to ZAP is a source-level transformation, accelerated by github.com/zap-proto/pb2zap — a syntactic go/ast codemod (a one-time migration tool, not a runtime bridge). For each construction site it:

1. Rewrites protobuf construction &x_pb.Req{F: v} into the ZAP build-once form xwire.NewReq(xwire.ReqInput{F: v}).
2. Adds the generated xwire import and drops the now-unused x_pb import.
3. Reports — never guesses — the read and mutate selectors it cannot mechanically transform (a field read becomes a View accessor; an in-place mutate has no ZAP equivalent and must be restructured by hand).

pb2zap moves the mechanical 80%; the reported read/mutate sites are finished by hand against the ZAP View/Input types. There is no automatic path for a mutate-in-place site because ZAP has no mutable message (§Rationale).

5. Conformance gate

The canonical, CI-enforceable gate is: zero first-party google.golang.org/grpc imports.

# MUST return no rows for a conforming repo:
grep -rl '"google.golang.org/grpc' --include='*.go' . | grep -v /vendor/

Verified empty for cloud, gateway, visor, and ai.

The gate is deliberately not grep google.golang.org/grpc go.mod. That command is non-empty and cannot be driven to zero while Hanzo depends on the upstream OpenTelemetry OTLP-HTTP exporter and the Gemini SDK, both of which list gRPC as an indirect requirement they never dial (§Residual & Conformance). A go.mod gate would be permanently red for reasons unrelated to whether any Hanzo code speaks gRPC. The import gate measures the real invariant: does first-party Hanzo code dial or serve gRPC? No.

The conformance gate is a grep over source, suitable for CI. A regression — a new dependency that imports google.golang.org/grpc first-party, or a "convenient" gRPC client added to a service — fails the gate deterministically, where a human reviewer might miss it in a large diff.

Residual & Conformance

Enforceable gate: PASS. First-party google.golang.org/grpc imports across cloud, gateway, visor, ai = zero (verified by the §5 grep). No Hanzo first-party code serves or dials gRPC.

Residual go.mod grpc is never-dialed transitive/indirect, and is not a conformance signal. Per repo:

Gate blind spot, stated honestly. The import grep catches direct first-party gRPC use; it does not catch a first-party import of a third-party SDK that internally dials gRPC to an external provider. The mandate governs Hanzo's own inter-service wire — a third-party cloud provider's control-plane SDK is interop, not a Hanzo transport. The governing rule: where a REST/HTTP surface exists, the gRPC SDK MUST be replaced (done for GCP compute in visor/ai); where none exists, the dependency is a documented exception that never targets a Hanzo service. No such exception exists in-tree today (verified: visor carries no external gRPC-dialing SDK).

Path to literal-zero go.mod (future work). Reaching an empty grep grpc go.mod requires eliminating the upstream transitive pulls, none of which is a Hanzo gRPC conversation:

1. ZAP-native metrics/logs exporter. Extend the zaptrace pattern (protowire hand-encode over zap-proto/http) to a metrics and logs exporter, retiring otlpmetrichttp/otlptracehttp — the OTLP-HTTP internal config that carries the indirect grpc in visor and ai.
2. Raw-HTTP Gemini client (ai). Replace google.golang.org/genai with a net/http client against the generativelanguage REST surface, removing the last direct dependency that transitively requires grpc.
3. Embedded O11y/GCS (cloud). The o11y collector is an OTLP/gRPC receiver by definition; its grpc is not a client dial and is out of scope for the client-side mandate. hanzoai/base GCS transport is a separate GCS- over-REST question tracked with the compute work.

Until then, the standard's conformance is measured where it is real and enforceable: zero first-party gRPC imports.

References

• HIP-0106: The Hanzo Plugin Contract — ZAP as the inter-subsystem contract; the canonical Hanzo Go stack; the "no .capnp in Hanzo-authored source" policy.
• HIP-0112: Cloud Topology — ingress → gateway → services; the external edge where HTTP/WS terminate and identity headers are minted.
• HIP-0114: ZAP — Inter-VM Cognitive Transport for Thinking Chains — ZAP as the inter-VM transport; "ZAP transports; proofs commit; receipts settle; VMs execute."
• github.com/zap-proto/{go,zip,http,ws,web,zapd,zap-spec} — the ZAP transport and framework ecosystem.
• github.com/zap-proto/pb2zap — the syntactic go/ast protobuf→ZAP migration codemod.
• OpenTelemetry OTLP specification — the span payload format ZAP carries unchanged (transport-substituted, not payload-modified).

6. Same machine is a unix socket, not a network hop

Cloud is one binary running plugins. When two of them sit on the same machine — which is the normal case, not the exception — a TCP hop to localhost is pure overhead: a syscall path, a checksum, a port, and a loopback round trip to reach a process sharing the same kernel.

ZAP over a unix socket is the PREFERRED transport for any plugin-to-plugin call inside cloud. Network ZAP remains correct where the peer is genuinely on another machine.

The rule, in order:

1. Same process — a direct call. No transport at all.
2. Same machine (plugin↔plugin, sidecar↔host) — ZAP over UDS.
3. Different machine — ZAP over the network.

A socket path is also an authorization boundary: filesystem permissions gate who may connect, with no port to scan and no interface to accidentally bind. A service reachable only through a socket in its own namespace cannot be reached from another pod at all.

Shipped: cloud/rpc.go — the internal plane over net.Listen("unix", path) with a 200ms dial probe to detect a live peer before binding, and dial.go/serve.go moving callers onto it (branch feat/zap-uds-internal-plane).

What UDS cannot do, so it is not claimed: a per-node DaemonSet reading /var/log/pods does not share a filesystem namespace with a central service. That path stays network ZAP unless the reader becomes co-located with its consumer. UDS is chosen because processes are co-located; it never makes them so.

Copyright

Copyright and related rights waived via CC0.