client/server: in-process dispatch fast-path (opt-in)#4854
Merged
Conversation
There was a problem hiding this comment.
Pull request overview
Adds an opt-in in-process dispatch fast-path for unary client calls when the target service is running in the same process and both request/response bodies are raw codec/bytes.Frame payloads. This aims to remove transport/dial overhead while still exercising the server router and codecs.
Changes:
- Introduces
internal/localdispatchas a neutral, process-local registry mapping service name → dispatcher function. - Registers/deregisters an
rpcServerlocal dispatcher onStart()/Stop()and serves requests via an in-memorytransport.Socket. - Adds
client.LocalDispatch()option plus tests/benchmarks and a changelog entry.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| server/rpc_server.go | Registers/deregisters the server’s local dispatcher on start/stop. |
| server/local.go | Implements the server-side in-process dispatch via a single-message in-memory socket. |
| internal/localdispatch/localdispatch.go | Adds the process-local dispatcher registry keyed by service name. |
| client/rpc_client.go | Adds an early local fast-path attempt before the network path. |
| client/options.go | Adds Options.LocalDispatch and the client.LocalDispatch() option helper. |
| client/local.go | Implements the client-side local dispatch lookup and message construction for raw frames. |
| client/local_test.go | Adds functional tests plus benchmarks for network vs local dispatch. |
| CHANGELOG.md | Documents the opt-in fast-path and benchmark numbers. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+59
to
+64
| func (s *rpcServer) localDispatch(ctx context.Context, req *transport.Message) (*transport.Message, error) { | ||
| contentType := req.Header["Content-Type"] | ||
| if contentType == "" { | ||
| contentType = DefaultContentType | ||
| req.Header["Content-Type"] = contentType | ||
| } |
Comment on lines
+86
to
+90
| // In-process fast-path: if the callee runs in this process and both bodies | ||
| // are raw frames, dispatch directly and skip the network entirely. | ||
| if handled, err := r.localCall(ctx, req, resp); handled { | ||
| return err | ||
| } |
Comment on lines
+105
to
+118
| // TestLocalDispatchFallsBackWhenNotLocal confirms a service not registered | ||
| // in-process still works via the network path even with LocalDispatch on. | ||
| func TestLocalDispatchFallsBackWhenNotLocal(t *testing.T) { | ||
| reg := registry.NewMemoryRegistry() | ||
| stop := startEchoServer(t, reg) | ||
| defer stop() | ||
|
|
||
| // LocalDispatch is on, but the call still resolves — the fast-path only | ||
| // engages when it fully applies, otherwise the network path runs. | ||
| local := newEchoClient(reg, client.LocalDispatch()) | ||
| if got := callEcho(t, local, "x").Msg; got != "echo:x" { | ||
| t.Fatalf("reply = %q, want echo:x", got) | ||
| } | ||
| } |
asim
force-pushed
the
claude/local-fastpath
branch
from
July 15, 2026 15:04
f1de1fa to
55efe62
Compare
When caller and callee run in the same process, a unary Call pays the full network tax — pool.Get, dial, codec-over-socket, and the transport pump — even though the handler table is right there. This adds an opt-in fast-path that dispatches directly. - internal/network: a neutral registry (transport.Message in/out) so client and server wire up without importing each other. A running server registers a dispatcher under its name on Start, deregisters on Stop. - server: localDispatch serves a request in-process through the same router (identical wrappers/codecs/error mapping) over an in-memory socket — no dial, no pipe, no gob. - client: LocalDispatch() opt-in. In call(), a unary request whose body and response are raw frames (codec/bytes.Frame — the agent/MCP/flow shape) dispatches locally; everything else falls back to the network path unchanged. Correctness test proves the fast-path returns byte-identical replies to the network path; benchmark shows ~545µs -> ~28µs (~20x) and ~3.6x fewer allocations. Off by default. Covers #4817 (path b); the zero-copy typed path remains a follow-up. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CmdEY7pYmV5zzwCjNJ4ykL
asim
force-pushed
the
claude/local-fastpath
branch
from
July 15, 2026 15:41
55efe62 to
3480874
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Covers #4817 (path b, codec-once) from the design proposal. Opening for review rather than auto-merging — it's the delicate core-dispatch item, and here are the promised numbers.
The problem
When caller and callee run in the same process, a unary
Callstill pays the full network tax —pool.Get→ dial → codec-over-socket → the transport pump — even though the server's handler table is right there. Mu measured ~398µs over HTTP loopback for exactly this.The change (opt-in, off by default)
internal/network— a neutral registry keyed by service name, speaking onlytransport.Message, soclientandserverwire up without importing each other (they don't today). A running server registers a dispatcher onStart, deregisters onStop.server.localDispatch— serves a request in-process through the same router (identical handler wrappers, codecs, and error mapping) over an in-memory socket — no dial, noio.Pipe, no gob.client.LocalDispatch()— the opt-in. Incall(), a unary request whose body and response are raw frames (codec/bytes.Frame— the shape agent / MCP / flow tool calls already use) dispatches locally; everything else falls back to the existing network path unchanged (disabled, streaming, non-frame bodies, or a service not in this process).Numbers (
go test -bench . ./client)≈20× faster, ~4.8× less memory, ~3.6× fewer allocations.
Correctness
TestLocalDispatchMatchesNetworkproves the fast-path returns a byte-identical reply to the same handler over the network path;TestLocalDispatchFallsBackWhenNotLocalconfirms fall-back. All existing./client/...and./server/...tests pass under-race;go vetandgolangci-lintclean. Because it's gated on an opt-in and only engages for the frame-passthrough shape, nothing changes for existing callers.Scope / follow-up
This is the codec-once path (b). The zero-copy typed path (a) — skipping the codec entirely for typed handlers when caller/callee share the process — is the natural follow-up once this seam is proven, and would push the ~28µs down further. Streaming stays on the network path by design.
🤖 Generated with Claude Code
https://claude.ai/code/session_01CmdEY7pYmV5zzwCjNJ4ykL