fix(watchdog): don't log optional Free() as an error when backend returns Unimplemented (#10602)#10607
Merged
mudler merged 2 commits intoJun 30, 2026
Conversation
…urns Unimplemented (mudler#10602) When the watchdog evicts a model, deleteProcess calls the backend's gRPC Free() to release VRAM before stopping the process. Free is optional: backends that don't override it -- the generated UnimplementedBackendServer stub, many Python/external backends, or a federation proxy in distributed mode -- return gRPC Unimplemented. That is expected, not a failure: VRAM is reclaimed when the local process is stopped, or by the remote unloader for remote backends. Logging it as "WARN Error freeing GPU resources" made a benign, optional RPC look like a fault (the alarming line in mudler#10602, seen in distributed mode where the model is remote and Free hits a stub). Treat gRPC Unimplemented from Free() as a no-op logged at Debug; genuine failures still Warn. Free() is still attempted for every backend, so any backend that does implement it is unaffected. Add a reusable grpcerrors.IsUnimplemented helper following the package's existing code-based detection idiom (prefer the typed status code, fall back to the message across non-gRPC boundaries), with table tests. Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Adira Denis Muhando <dennisadira@gmail.com>
mudler
reviewed
Jun 30, 2026
| if grpcerrors.IsUnimplemented(err) { | ||
| xlog.Debug("Backend does not implement Free(); GPU release handled on process stop", "model", s) | ||
| } else { | ||
| xlog.Warn("Error freeing GPU resources", "error", err, "model", s) |
Owner
There was a problem hiding this comment.
what about moving to error at this point? It was a warning before because we were not doing any distinction, but we could treat all the others as real error now
Contributor
Author
There was a problem hiding this comment.
Good call — done in 8e520dd. Since the expected Unimplemented case is now split out to Debug, the remaining branch is a genuine failure to release VRAM, so it's logged at error level now.
Per review: now that the expected gRPC Unimplemented case is split out and logged at Debug, any remaining Free() error is a genuine failure to release VRAM, so surface it at error level instead of warn. Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Adira Denis Muhando <dennisadira@gmail.com>
mudler
approved these changes
Jun 30, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fixes #10602.
In distributed mode, the frontend container logged
WARN Error freeing GPU resources error=rpc error: code = Unimplementedwhen the watchdog evicted a remote model overnight.Root cause
When the watchdog evicts a model,
ModelLoader.deleteProcesscalls the backend's gRPCFree()to release VRAM before stopping the process.Freeis optional: backends that don't override it return gRPCUnimplemented— this includes the generatedUnimplementedBackendServerstub, many Python/external backends, and a federation proxy in distributed mode (where the model is remote,process == nil, and freeing is actually done by theremoteUnloader).That
Unimplementedresponse is expected, not a fault — VRAM is reclaimed when the local process is stopped, or by the remote unloader for remote backends. Logging it asWARN Error freeing GPU resourcesmade a benign, optional RPC look like a failure.Change
pkg/grpc/grpcerrors: add reusableIsUnimplemented(err)helper, following the package's existing code-based detection idiom (prefer the typed gRPC status code, fall back to the message across non-gRPC boundaries). Table tests included.pkg/model/process.go:Free()returningUnimplementedis now logged atDebug("backend doesn't implement Free; GPU release handled on process stop"); genuine failures stillWarn.Free()is still attempted for every backend, so backends that do implement it are unaffected.Testing
go test ./pkg/grpc/grpcerrors/...andgo test ./pkg/model/...pass.gofmt/go vetclean on the changed files.🤖 Generated with Claude Code