fix: erase persistent_term leak in GRPC.Client.Connection on disconnect#509
Open
ryochin wants to merge 1 commit intoelixir-grpc:masterfrom
Open
fix: erase persistent_term leak in GRPC.Client.Connection on disconnect#509ryochin wants to merge 1 commit intoelixir-grpc:masterfrom
ryochin wants to merge 1 commit intoelixir-grpc:masterfrom
Conversation
Each connect/disconnect cycle leaks one persistent_term entry because the entry is never erased. Since persistent_term is not garbage-collected, this causes unbounded memory growth on long-running nodes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
GRPC.Client.Connection.init/1stores a persistent_term entry keyed by the channel's ref:On disconnect,
handle_call({:disconnect, _})drops thevirtual_channelfrom state viaMap.drop/2, then stops the GenServer via{:continue, :stop}:However, the persistent_term entry is never erased -- neither in the disconnect handler nor in
terminate/2(which is a no-op:def terminate(_reason, _state), do: :ok).By default, each
connect/2call generates a fresh ref viamake_ref(), every connect/disconnect cycle permanently leaks one persistent_term entry.Impact
Applications that create short-lived connections (e.g. connect -> RPC -> disconnect per request) accumulate persistent_term entries indefinitely. Unlike regular process memory, persistent_term entries are not garbage-collected and persist for the lifetime of the BEAM node, causing steady memory growth with no upper bound.
Proposed fix
Erase the persistent_term entry in
handle_call({:disconnect, ...}), beforeMap.dropremoves the ref from state:Additionally,
terminate/2should erase the entry as a safety net for abnormal termination paths wheredisconnectis never called:Note:
terminate/2alone is insufficient becauseMap.drop(state, [:real_channels, :virtual_channel])removes the ref from state beforeterminateis called in the normal disconnect path. Both locations are needed.Reproduction
Requires a gRPC server listening on
localhost:50051(any service will do).