Add Embabel AgentPlatform integration plan for agentic aggregates#301
Conversation
Agent-Logs-Url: https://github.com/elasticsoftwarefoundation/akces-framework/sessions/cf4ce3d9-2ec1-43d1-b86a-dd9335f4a85a Co-authored-by: jwijgerd <914840+jwijgerd@users.noreply.github.com>
…: agentHandledCommands, agentHandledEvents, agentProducedErrors Agent-Logs-Url: https://github.com/elasticsoftwarefoundation/akces-framework/sessions/4c3ee463-b499-461a-bf05-e5c611164cf5 Co-authored-by: jwijgerd <914840+jwijgerd@users.noreply.github.com>
…el-integration-plan.md) Agent-Logs-Url: https://github.com/elasticsoftwarefoundation/akces-framework/sessions/42af408a-7ba0-4aa8-9862-bba2fb216adf Co-authored-by: jwijgerd <914840+jwijgerd@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Adds a consolidated Embabel AgentPlatform integration plan for the agentic aggregates module, and removes the prior plan location.
Changes:
- Introduces a new integration plan document covering
AgentPlatformwiring, agent-handled commands/events, and agent-produced error events. - Documents phased implementation steps (runtime injection, partition integration, Embabel components, and testing).
- Removes the legacy
embabel-integration/plan directory (verified absent in the repo).
| **Changes**: | ||
| - In `AgenticAggregateRuntimeFactory.getObject()`, look up the `AgentPlatform` bean from `applicationContext` | ||
| - Pass it as a constructor argument to `new KafkaAgenticAggregateRuntime(kafkaRuntime, objectMapper, stateClass, agentPlatform)` | ||
| - Handle the case where `AgentPlatform` is not available (e.g., during unit tests without Embabel auto-configuration): fall back to a null reference with a warning log, making agent-based command processing a no-op |
There was a problem hiding this comment.
The AgentPlatform not being available should be a fatal error, the application shold not start
There was a problem hiding this comment.
Updated in a1d02e4 — AgentPlatform unavailability is now a fatal error. The plan specifies: "fail fast with a fatal error — the application must not start without a configured AgentPlatform."
|
|
||
| ### 2.2 Memory Management Action: `StoreMemoryAction` | ||
|
|
||
| **What**: An `@Action`-annotated method that creates a `StoreMemoryCommand` from the current `AgentProcess` context and sends it to the aggregate's command bus. |
There was a problem hiding this comment.
I want to refactor out the use of internal commands for memory management. The Embabel actions should produce events that are applied to the state (using the built-in EventSourcingHandlers in this case). The commands will be removed in a later stage
There was a problem hiding this comment.
Updated in a1d02e4 — StoreMemoryAction now produces MemoryStoredEvent directly (applied via the built-in @EventSourcingHandler which calls MemoryAwareState.withMemory()). Added a note that StoreMemoryCommand will be removed in a later stage.
|
|
||
| ### 2.3 Memory Management Action: `ForgetMemoryAction` | ||
|
|
||
| **What**: An `@Action`-annotated method that creates a `ForgetMemoryCommand` to remove a specific memory by ID or by criteria. |
There was a problem hiding this comment.
Same thing here, return the event not the command
There was a problem hiding this comment.
Updated in a1d02e4 — ForgetMemoryAction now produces MemoryRevokedEvent directly (applied via the built-in @EventSourcingHandler which calls MemoryAwareState.withoutMemory()). ForgetMemoryCommand noted for later removal.
|
|
||
| **Why**: Some actions only make sense when the agent has prior context. For example, "recall memories" is only meaningful when memories exist. | ||
|
|
||
| ### 2.6 Condition: `HasAggregateStateCondition` |
There was a problem hiding this comment.
In another feature we will implement a mechanism where the framework will ensusre that for an AgenticAggregate the State is always created. This makes sense because it is a singleton. So this condition can be removed
There was a problem hiding this comment.
Updated in a1d02e4 — HasAggregateStateCondition (section 2.6) is removed with a note that the framework ensures singleton state is auto-created. Also removed hasAggregateState from the blackboard conditions in section 4.3.
|
|
||
| ## Open Questions for Architect | ||
|
|
||
| 1. **LLM Model Configuration**: Should the `@AgenticAggregateInfo` annotation include LLM model preferences (model name, temperature), or should this be purely configuration-based (application.yaml)? |
There was a problem hiding this comment.
The LLM models should be configured at the Embabel level, so not in the AgenticAggregateInfo. So indeed configuration based or in the Emabel code / annotations
There was a problem hiding this comment.
Updated in a1d02e4 — Resolved as: LLM models are configured at the Embabel level (Embabel code/annotations or application configuration), not in @AgenticAggregateInfo.
| ## Open Questions for Architect | ||
|
|
||
| 1. **LLM Model Configuration**: Should the `@AgenticAggregateInfo` annotation include LLM model preferences (model name, temperature), or should this be purely configuration-based (application.yaml)? | ||
| 2. **Agent Scope**: Should each agentic aggregate get its own `Agent` instance (deployed to the `AgentPlatform`), or should the `AkcesAgentComponent`'s actions be shared across all agentic aggregates? |
There was a problem hiding this comment.
Each AgenticAggregate gets one Agent instance, so when multiple AgenticAggregates are deployed in a single VM instance (sharing the AgentPlatform) the all use their own Agent instance.
There was a problem hiding this comment.
Updated in a1d02e4 — Resolved as: Each AgenticAggregate gets its own Agent instance. Multiple aggregates in a single VM share the AgentPlatform but each uses its own Agent.
|
|
||
| 1. **LLM Model Configuration**: Should the `@AgenticAggregateInfo` annotation include LLM model preferences (model name, temperature), or should this be purely configuration-based (application.yaml)? | ||
| 2. **Agent Scope**: Should each agentic aggregate get its own `Agent` instance (deployed to the `AgentPlatform`), or should the `AkcesAgentComponent`'s actions be shared across all agentic aggregates? | ||
| 3. **Synchronous vs. Async Agent Execution**: Should `agentProcess.run()` be called synchronously within the Kafka poll loop, or should we use `agentProcess.tick()` for incremental execution with periodic Kafka heartbeats? |
There was a problem hiding this comment.
It should be possible for an AgenticAgent to have multiple running processes in the future. Currently we don't have support for this (we would need to add more state etc). For now use AgentProcess.tick() but if there is an active AgentProcess we must call tick() even if there are no records coming back from the kafka poll
There was a problem hiding this comment.
Updated in a1d02e4 — Resolved as: Use AgentProcess.tick() for incremental execution. tick() must be called on every poll loop iteration if there is an active process, even without Kafka records. Multiple processes per agent noted as future scope. Updated the result translation section (4.2) accordingly.
| 1. **LLM Model Configuration**: Should the `@AgenticAggregateInfo` annotation include LLM model preferences (model name, temperature), or should this be purely configuration-based (application.yaml)? | ||
| 2. **Agent Scope**: Should each agentic aggregate get its own `Agent` instance (deployed to the `AgentPlatform`), or should the `AkcesAgentComponent`'s actions be shared across all agentic aggregates? | ||
| 3. **Synchronous vs. Async Agent Execution**: Should `agentProcess.run()` be called synchronously within the Kafka poll loop, or should we use `agentProcess.tick()` for incremental execution with periodic Kafka heartbeats? | ||
| 4. **MCP Tool Scoping**: Should MCP tools (from sidecar servers) be automatically added to the agent's tool groups, or should they be explicitly configured per agentic aggregate? |
There was a problem hiding this comment.
The MCP tools added in the core AgenticAggregate should be available to all Agents
There was a problem hiding this comment.
Updated in a1d02e4 — Resolved as: MCP tools from the core AgenticAggregate are available to all Agents. Shared tools are the default, no per-aggregate scoping needed.
| 2. **Agent Scope**: Should each agentic aggregate get its own `Agent` instance (deployed to the `AgentPlatform`), or should the `AkcesAgentComponent`'s actions be shared across all agentic aggregates? | ||
| 3. **Synchronous vs. Async Agent Execution**: Should `agentProcess.run()` be called synchronously within the Kafka poll loop, or should we use `agentProcess.tick()` for incremental execution with periodic Kafka heartbeats? | ||
| 4. **MCP Tool Scoping**: Should MCP tools (from sidecar servers) be automatically added to the agent's tool groups, or should they be explicitly configured per agentic aggregate? | ||
| 5. **Error Event Validation at Runtime**: Should the `AgentProcessResultTranslator` strictly validate that emitted error events are in the `agentProducedErrors` set, or should it also accept system error events without explicit declaration? |
There was a problem hiding this comment.
We should accept all ErrorEvents, even if they are not declared (log a warning if this happens)
There was a problem hiding this comment.
Updated in a1d02e4 — Resolved as: Accept all ErrorEvents at runtime, even undeclared ones. Log a warning for undeclared error events but don't reject. Updated the result translation section (4.2) and integration test descriptions accordingly.
…mands for memory, remove HasAggregateStateCondition, add memory limits, resolve design decisions Agent-Logs-Url: https://github.com/elasticsoftwarefoundation/akces-framework/sessions/e39a68f0-0493-4be8-99ab-d03126aa1a11 Co-authored-by: jwijgerd <914840+jwijgerd@users.noreply.github.com>
jwijgerd
left a comment
There was a problem hiding this comment.
In general I am missing a mechanism on applying DomainEvent instances created from the Embabel runtime. Find a solution where after each process step (i.e. each call of AgentProcess.tick() any events that are generated are collected an applied. This should be done in the Akces code. It looks like we need to expose the (now private) processEvent methods in the KafkaAggregateRuntime. Maybe it is a better idea to extend KafkaAggregateRuntime (and make the methods protected) instead of using it as a delegate? Consider this solution, but only if it makes sense
| | `AgentHandledCommandsRegistrationTest` | Verifies `agentHandledCommands` create `CommandType` entries without handler adapters | | ||
| | `AgentHandledEventsRegistrationTest` | Verifies `agentHandledEvents` create external `DomainEventType` entries | | ||
|
|
||
| ### 5.2 Integration Tests |
There was a problem hiding this comment.
create a test agent implementation, base it on the example you give in the beginning of the plan
There was a problem hiding this comment.
Added section 5.2 "Test Agent Implementation: TestTradingAdvisorAgent" in 7de97ce. Includes a full TestTradingAdvisorAggregate (with deterministic @CommandHandler for creation + agent-handled TestAnalyzeMarketCommand), TestTradingAdvisorState implementing MemoryAwareState, all command/event/error event classes, and a TestTradingAdvisorAgentComponent @EmbabelComponent with test actions. Also added integration test cases for the full agent loop, external events, and the event application mechanism.
… to plan Agent-Logs-Url: https://github.com/elasticsoftwarefoundation/akces-framework/sessions/8b649eeb-18cb-4c8a-b3a2-8b515fe6690b Co-authored-by: jwijgerd <914840+jwijgerd@users.noreply.github.com>
Added section 4.3 "Event Application Mechanism" in 7de97ce. The plan proposes refactoring |
|
|
||
| --- | ||
|
|
||
| ## Phase 3: Akces Platform Development Goals, Actions, and Conditions |
There was a problem hiding this comment.
Add a way to discover other aggregates in the system (by exposing the AggregateServiceRecords) this includes other AgenticAggregates. We need this to be able to determine what commands we can send to achieve a Goal
There was a problem hiding this comment.
Added DiscoverAggregateServicesAction (section 3.6) in 85ec129. The AggregateServiceRecords from the Akces-Control topic are placed on the blackboard by the agentic handler adapters before agent invocation. The action exposes aggregate name, command topic, supported commands, produced events, consumed events, and type (STANDARD/AGENTIC) — enabling the agent to discover what commands can be sent to other aggregates to achieve a Goal.
|
|
||
| ### Registration Flow for Agent Properties | ||
|
|
||
| In `AgenticAggregateBeanFactoryPostProcessor`: |
There was a problem hiding this comment.
Would it not be possible to create new implementations of the CommandHandlerFunctionAdapter and the EventHandlerFunctionAdapter for the Agentic handlers? We can then already inject the AgentPlatform (and other resources needed) and already know we are handling a Command or a DomainEvent in the Embabel agent. We could the also setup the Blackboard etc in this implementation. Does this make sense?
There was a problem hiding this comment.
Reworked the plan in 85ec129 to use AgenticCommandHandlerFunctionAdapter and AgenticEventHandlerFunctionAdapter. The adapters implement CommandHandlerFunction / EventHandlerFunction, so they plug directly into the existing KafkaAggregateRuntime.handleCommand() flow. Inside apply(), the adapter sets up the Blackboard (command/event, state, memories, AggregateServiceRecords), creates the AgentProcess, calls tick(), and returns the resulting events as Stream<DomainEvent>. This eliminates the need to make processDomainEvent protected or refactor to inheritance — the delegation pattern stays as-is.
…pattern - Add DiscoverAggregateServicesAction (section 3.6) exposing AggregateServiceRecords to enable agents to discover other aggregates and their supported commands - Rework registration flow to use AgenticCommandHandlerFunctionAdapter and AgenticEventHandlerFunctionAdapter instead of bypassing handlers - Adapters plug into existing CommandHandlerFunction/EventHandlerFunction contracts, routing through the Embabel agent inside apply() - Simplify event application mechanism (section 4.3): no need to make processDomainEvent protected or refactor to inheritance - Update blackboard content (section 4.4) to include aggregate service records - Update file inventory and test descriptions Agent-Logs-Url: https://github.com/elasticsoftwarefoundation/akces-framework/sessions/f31ba8d2-cb16-4310-95ea-e7af6584122a Co-authored-by: jwijgerd <914840+jwijgerd@users.noreply.github.com>
| - Create a `AgentProcessResultTranslator` utility class to handle this translation | ||
| - Accept all `ErrorEvent` instances at runtime, even if not declared in `agentProducedErrors`. Log a warning for undeclared error events but do not reject them. | ||
|
|
||
| **Note on `tick()` execution**: Use `AgentProcess.tick()` for incremental execution. If there is an active `AgentProcess`, `tick()` must be called on every poll loop iteration, even if no Kafka records are returned. |
There was a problem hiding this comment.
We can only call tick() once in the handler and then we need to return control to the Akces runtime. This exposes a problem where we need to know if there are running AgentProcess instances that need another tick. Idea: we extend the handles with a new interface: AgenticProcessHandler. This interface also has a proceed(AgenticState) method that also returns DomainEvents and does a similar loop as the apply method. Currently we have no notion in the state of running Agent Tasks so for now implement it by calling the AgentProcess.tick() method until the end state. We will change this later when we add the AgentTask notion to the state (done in a separate feature)
There was a problem hiding this comment.
Updated in 8eac209 — Added AgenticProcessHandler interface with proceed(S state) returning Stream<E>, implemented on both AgenticCommandHandlerFunctionAdapter and AgenticEventHandlerFunctionAdapter. Current implementation calls tick() in a loop until end state inside apply(). The proceed() method is stubbed for future use when AgentTask support is added to the state, at which point apply() will call tick() only once and proceed() will handle subsequent ticks on active tasks.
…ntic handler adapters - Both AgenticCommandHandlerFunctionAdapter and AgenticEventHandlerFunctionAdapter now implement AgenticProcessHandler<S, E> interface - Interface defines proceed(state) method for future incremental tick support - Current implementation: apply() calls tick() in loop until end state (temporary) - Future: single tick per invocation with proceed() for active AgentTask advancement - Added AgenticProcessHandler.java to file inventory - Updated sections 4.1, 4.2, 4.3 for consistency with tick-to-completion approach Agent-Logs-Url: https://github.com/elasticsoftwarefoundation/akces-framework/sessions/eac8c191-eeee-47ad-b736-2fd859322b62 Co-authored-by: jwijgerd <914840+jwijgerd@users.noreply.github.com>
…dAgentTasks() to plan - Added AgentTask record with String id field to the Agentic Handler Adapters section - Described activeAgentTasks property on agentic aggregate state - Updated AgenticProcessHandler.proceed() signature from proceed(state) to proceed(AgentTask, state) - Updated both adapter implementations (command and event) with new proceed(AgentTask, state) signature - Added section 4.3.1 proceedAgentTasks() on KafkaAggregateRuntime - Added AgenticAggregatePartition flow showing proceedAgentTasks() called when no Kafka records - Updated all forward references (sections 4.1, 4.2, 4.3) to use new signatures - Added AgentTask.java to file inventory Agent-Logs-Url: https://github.com/elasticsoftwarefoundation/akces-framework/sessions/3cc284de-0f6f-4e06-85c3-c30a423df340 Co-authored-by: jwijgerd <914840+jwijgerd@users.noreply.github.com>
plans/embabel-integration-plan.mdDiscoverAggregateServicesAction(section 3.6)AgenticCommandHandlerFunctionAdapter/AgenticEventHandlerFunctionAdapter)AgenticProcessHandlerinterface implemented on both agentic handler adaptersAgentTaskrecord withString idfield, tracked in state'sactiveAgentTaskslistproceed(state)→proceed(AgentTask, state)inAgenticProcessHandlerand both adapter implementationsproceedAgentTasks()method onKafkaAggregateRuntime, called fromAgenticAggregatePartitionwhen no Kafka records are returnedproceedAgentTasks()flow for active task advancement