[Feature Request] Add context window management and conversation summarization support

[jeanlopezxyz]

Summary

When using kagent with models that have limited context windows (e.g., gpt-4o-mini with 128k tokens), agents with many MCP tools can exceed the context limit, causing "Session terminated" errors.

Currently, kagent does not expose ADK's context management features through the Agent CRD.

Problem

With an agent configured with ~40+ MCP tools, the context quickly exceeds limits:

• Each tool definition adds tokens to the context
• Conversation history accumulates
• No way to truncate, summarize, or manage context programmatically

Error observed:

context_length_exceeded: This model's maximum context length is 128000 tokens. 
However, your messages resulted in 155XXX tokens.

Proposed Solution

Expose ADK's context management capabilities through the Agent CRD:

Option 1: Context Compression (Compaction)

ADK has Context Compression that summarizes older events:

apiVersion: kagent.dev/v1alpha2
kind: Agent
spec:
  declarative:
    contextCompression:
      enabled: true
      compactionInterval: 5  # Compress every 5 events
      overlapSize: 1
      summarizer:
        type: llm  # Use LlmEventSummarizer
        model: gpt-4o-mini

Option 2: Context Caching

ADK has Context Caching for optimizing repeated requests:

spec:
  declarative:
    contextCache:
      enabled: true
      minTokens: 1000
      ttlSeconds: 3600
      cacheIntervals: 10

Option 3: Context Window Compression Config

From the ADK API, there's ContextWindowCompressionConfig:

spec:
  declarative:
    contextWindowCompression:
      enabled: true
      maxLength: 100000  # Max context length before compression

Alternatives Considered

1. Reduce tools: Works but limits agent capabilities
2. Use larger context models: More expensive (gpt-4o vs gpt-4o-mini)
3. Memory/RAG: Currently not supported in ADK (v0.6 release notes confirm this)

Additional Context

• ADK already implements these features, they just need to be exposed in the CRD
• This is critical for production deployments with complex agents
• ADK documentation: https://google.github.io/adk-docs/context/

Environment

• kagent version: v0.7.6
• Kubernetes: OpenShift 4.17
• Model: gpt-4o-mini (128k context)
• Tools: 16 MCP tools (reduced from 42 to mitigate issue)

[Insomniac2904]

hey @jeanlopezxyz , sorry i did not make a design doc . Would you like me to make short summary of the made changes?

[ronbutbul]

any updates on that matter?
(compression and caching feature)

can not really work like this currently
im hitting token limit very quickly cause every prompt history is being added to the context

[m13v]

i run 28 MCP servers in my daily setup, roughly 400 tools total. the token overhead from tool schemas alone is around 50k tokens before a single message is sent. the part nobody talks about is that context compression doesn't actually solve the root problem, it just delays it. tool definitions are injected on every request, they're not part of conversation history, so compaction of older events won't touch them. what actually worked for me was aggressive tool filtering at the agent config level, giving each subagent only the 5-8 tools it actually needs instead of the full set. went from constant context overflows to zero in about a week. the tradeoff is you need to think upfront about which tools each agent role requires, but that's a design problem not a runtime one. caching helps with the conversation side but the schema bloat is the real killer at 40+ tools.

[danialkhatib]

any updates on that matter?