diff --git a/SUMMARY.md b/SUMMARY.md index 8ddb564..9dd54d8 100644 --- a/SUMMARY.md +++ b/SUMMARY.md @@ -113,3 +113,8 @@ ## Looking for a home * [vision/talus-vision/explorer.md](vision/talus-vision/explorer.md) +* [nexus/adr/address-balance-gas-payment.md](nexus/adr/address-balance-gas-payment.md) +* [nexus/adr/auto-abort-expired-executions.md](nexus/adr/auto-abort-expired-executions.md) +* [nexus/adr/leader-status-ownership-versioning.md](nexus/adr/leader-status-ownership-versioning.md) +* [nexus/adr/multi-coin-gas-payment.md](nexus/adr/multi-coin-gas-payment.md) +* [nexus/packages/reference/nexus_registry/agent_registry.md](nexus/packages/reference/nexus_registry/agent_registry.md) diff --git a/nexus/TAP/agent-development.md b/nexus/TAP/agent-development.md index f990a51..c1d0f12 100644 --- a/nexus/TAP/agent-development.md +++ b/nexus/TAP/agent-development.md @@ -1,36 +1,70 @@ # Agent Development -Nexus, as an agentic framework, provides a useful set of abstractions and, both onchain and offchain components that orchestrate the development, registration and discovery of tools, the composition of those tools in workflows and finally the workflow execution. Find the definitions of these terms in the [glossary](../glossary.md) in case they are unfamiliar. - -As a result, Nexus greatly facilitates building Talus Agents for [agent developers](../index.md#actors). +Nexus, as an agentic framework, provides on-chain and off-chain components for developing agents, registering skills, discovering endpoints, composing tools into workflows, and executing those workflows. Find the definitions of these terms in the [glossary](../glossary.md) if they are unfamiliar. {% hint style="info" %} -A Talus agent is an onchain identity associated with one or more onchain assets and workflow services +A Talus agent is a standard TAP `Agent` identity registered in the TAP registry and associated with one or more workflow services, endpoint revisions, payment policies, and on-chain assets. {% endhint %} -## Development Requirements +## Standard TAP Model + +New TAP development uses the standard TAP registry and endpoint model in `nexus_interface::tap`. The legacy witness-based default TAP runtime is retired for active authoring; see [Default TAP Retirement](default-tap.md) for compatibility context. -There's a number of ways an agent developer could build their agents, with varying levels of complexity and customization: +The standard model has these core objects: -* **Custom TAP + custom workflow (hard)**: Construct the workflow(s) (DAGs) AND write a Sui Move "Talus Agent Package" (TAP) that holds the workflows(s) as well as some custom business logic related to it. This is the most complex as it requires the agent developer to develop a custom TAP package and define the DAG. In return you get the highest amount of customization. +* `Agent`: the keyed on-chain identity for the agent. +* `AgentSkill`: an agent-owned skill child addressed by an agent-local `SkillId` (`u64`). +* endpoint revisions: versioned execution metadata for a skill, including endpoint object metadata, shared object requirements, and activation state. +* `DefaultDagExecutor`: the registry-backed default runtime-selected DAG target used by default DAG execution flows. +* `AgentPaymentVault`: the agent-owned vault created for every standard Talus agent. +* `ExecutionPayment`: the execution-bound payment child used for immediate and scheduled TAP execution. -* **Custom TAP + existing workflow (medium)**: Write a Sui Move TAP that uses existing workflows in a _plug-and-play_ manner. There could be additional business logic added to the TAP. This is an intermediate level of complexity that gives a sufficient amount of customization provided there are workflows available that satisfy requirements. +## Development Paths -* **Default TAP + custom workflow (easy)**: Only construct the workflow(s) (DAGs) and use the [_default TAP_](default-tap.md) to execute it and call the workflow engine. The default TAP provides a template TAP implementation and reduces the development work to the configuration of the workflow DAG(s). This simplifies development considerably at the cost of flexibility in adding business logic to trigger DAG execution. +* **Registered agent skill**: Create an `Agent`, publish or select a DAG, register a skill with `register_skill`, and activate endpoint revisions through the TAP registry. This is the default path for application agents. +* **Default runtime DAG executor**: Use the deployment-provided default DAG executor for runtime-selected DAG execution when no app-specific TAP business logic is needed. +* **Custom TAP package + standard registry**: Publish a package with custom endpoint objects or business logic, but still register the agent, skills, endpoint revisions, authorization schema, and payments through the standard TAP registry and interface. {% hint style="success" %} -How to choose the way to build? +Choose the simplest path that preserves the authorization and payment behavior your agent needs. Most agents should start with a standard registered skill and add custom TAP package logic only when endpoint state or business rules require it. +{% endhint %} -The choice of how to build the agent will be determined by weighing: +## Standard Procedure -* familiarity with Sui Move, -* the need for custom logic to govern workflow execution, payment etc. -* the availability of existing workflows that meet the application's needs -{% endhint %} +1. Publish the workflow DAG or choose runtime-selected default DAG execution. +2. Define the skill requirements: input schema hash, workflow hash, metadata hash, `TapPaymentPolicy`, `TapSchedulePolicy`, and `TapVertexAuthorizationSchema`. +3. Create the agent through the TAP registry. Agent creation also creates the agent's `AgentPaymentVault`. +4. Register the skill with the owning `Agent`. `register_skill` allocates the next agent-local `SkillId`, stores the `AgentSkill` child under the agent, stores registry records, and writes the initial endpoint revision. +5. Publish or provide the endpoint object for the TAP package when the skill needs endpoint-local state. The endpoint object ID, version, digest, shared object references, interface revision, and requirements are committed through the endpoint config digest. +6. Announce later endpoint revisions with `announce_endpoint_revision` and activate the revision intended for new executions with `set_active_endpoint_revision`. +7. Publish discovery metadata by relying on standard TAP events and registry records. Leaders and SDK clients resolve active runtime state by `(agent_id, skill_id, interface_revision)`. +8. Execute through the registered agent path or the default runtime DAG executor path. A worksheet pins the `Agent`, `SkillId`, endpoint revision, endpoint object, and execution ID so later payment and authorization checks use the same revision. +9. Create payment using the endpoint policy. Invoker-funded execution supplies a payment coin; agent-funded execution uses the `AgentPaymentVault`. Both paths create an execution-bound `ExecutionPayment` child under the `DAGExecution`. +10. Finalize payment after execution. Successful executions accomplish the payment; failed or rejected executions refund it. Scheduled occurrences finalize against the occurrence payment and then update the scheduled task state. + +## Default Runtime DAG Execution + +The current default path is a registry-owned default agent target, not the old witness `nexus_workflow::default_tap` module. Deployment bootstraps a runtime-selected default skill and stores a `DefaultDagExecutor` in the TAP registry. Default DAG execution and scheduler-triggered default execution resolve that target, create the same TAP worksheet/payment context, and pin the runtime-selected DAG in execution evidence. + +Use this path when the agent does not need package-specific endpoint logic. Use a registered skill when the agent needs a stable agent-owned skill with its own endpoint revisions, payment policy, schedule policy, or authorization schema. + +## Payment Vaults + +Every standard Talus agent receives an `AgentPaymentVault` when the agent is created. The vault is the canonical on-chain balance holder for agent-funded explicit agent execution. + +Anyone can deposit SUI into an agent vault. Withdrawals are restricted to the agent owner or operator through the standard TAP registry authorization path. When an endpoint policy selects agent-funded execution, TAP payment creation locks the requested budget from the agent vault into an `ExecutionPayment`. Consumption is tracked on the payment, and finalization either charges the consumed amount on accomplish or releases the lock on refund. + +This does not force every TAP endpoint to use agent funds. `TapPaymentPolicy` determines whether execution is invoker-funded or agent-funded, and scheduled execution can use address-funded reserve or agent-vault-funded reserve depending on the selected agent path. + +## Scheduling and Authorization + +Scheduled agent execution creates a `ScheduledSkillTask` tied to the agent, skill, endpoint revision, payment source, reserve, and occurrence policy. Each triggered occurrence converts prepaid reserve into a normal execution-bound payment, then completion records whether that occurrence accomplished or refunded and whether the task may continue recurring. + +Fixed on-chain tool authorization uses `TapVertexAuthorizationSchema` and per-vertex authorization grants. Endpoint revisions commit to the allowed fixed tools and payment requirement. Leaders verify the grant against the pinned worksheet before a one-use `VertexAuthorizationCheckCap` can authorize the fixed tool call. -## Explore further +## Explore Further -Continue learning about agent development with the following sections: +Continue learning about agent development with: -* the [Nexus interface for TAPs](../packages/nexus-interface.md) section, outlining what interface the TAP must comply with -* the reference [default TAP](default-tap.md) implementation, that serves as an example for a bare-bones TAP \ No newline at end of file +* the [Nexus interface for TAPs](../packages/nexus-interface.md), which summarizes the active standard TAP interface surface +* [Default TAP Retirement](default-tap.md), which explains the retired witness path and the standard default DAG executor replacement diff --git a/nexus/TAP/default-tap.md b/nexus/TAP/default-tap.md index 489cd12..c923a1b 100644 --- a/nexus/TAP/default-tap.md +++ b/nexus/TAP/default-tap.md @@ -1,364 +1,18 @@ -# Default Talus Agent Package (TAP) +# Default TAP Retirement -## Overview +The legacy witness-based `nexus_workflow::default_tap` module has been +retired as an active runtime and public authoring path. -The Default TAP is a useful helper component for Nexus agent developers. It serves as a template or base implementation of a Talus Agent Package (TAP) that can be used for examples, tests, or to run JSON-defined DAGs from the CLI. +Use the standard TAP registry, standard endpoint object, and +`default_tap_target` metadata instead. New executions are routed through +standard Talus agent, skill, endpoint, payment, agent-vault, and authorization +context. -## Interface Compliance +New standard Talus agents always receive a shared `AgentPaymentVault`. Default +target execution can still use invoker-funded payment, but agent-triggered or +scheduled flows can select agent-vault-backed payment when the endpoint policy +allows it. -The Default TAP implements the [Nexus Interface V1][nexus-interface-v1] specification, which defines the required functionality for any Talus Agent Package to integrate with the Nexus workflow engine. Key interface requirements include: - -1. **Version Management** - - Must declare and maintain interface version compatibility. - - Must support version checking for backward compatibility. - -1. **Workflow Management** - - Must handle worksheet management and state tracking. - - Must support tool evaluation confirmation. - -1. **Authorization** - - Must implement witness-based authorization. - - Must support package upgrade mechanisms. - -For detailed interface requirements, see the [Nexus Interface Documentation][nexus-interface]. - - - -{% hint style="info" %} -In the code snippets below, we reference some Sui Move patterns (e.g. hot potato), please refer to the [primitives package doc][primitives] for more information on the approach taken here. -{% endhint %} - -### DefaultTAP Structure - -The agent will be represented by the `DefaultTAP` struct, which is a shared object that contains: - -- `id`: A unique identifier for the TAP instance. -- `witness`: A Bag containing authorization tokens for package identification and upgrade management. -- `iv`: The interface version (currently v1) that clients can use to determine compatibility. - -```rust -use nexus_interface::version::InterfaceVersion; -use sui::bag::{Self, Bag}; -use sui::object::UID; - -/// Shared object implementing the Nexus Interface v1 specification. -public struct DefaultTAP has key { - id: UID, - /// On package upgrade, we'll want to replace the previous witness with a - /// new one that identifies the new package. - /// - /// This is because as of now Sui doesn't give us access to the package ID - /// so we need to always create a new type and query it from the type via - /// its type name. - witness: Bag, - /// Clients can search the talus agent shared object for this type to - /// determine what interface to expect. - iv: InterfaceVersion, -} -``` - -### Constructor and Leader Registration - -The DefaultTAP is created using the `new()` constructor function, which: - -1. Creates a new DefaultTAP instance with a unique ID. -1. Initializes a witness token for package identification. -1. Sets the Nexus interface version to v1. -1. Registers the TAP with the Nexus leader. - -```rust -use nexus_interface::version::{Self, InterfaceVersion}; -use sui::bag::{Self, Bag}; -use sui::object::UID; -use sui::transfer::share_object; - -public(package) fun new(ctx: &mut TxContext) { - let default_tap = DefaultTAP { - id: object::new(ctx), - witness: { - let mut b = bag::new(ctx); - b.add(b"witness", DefaultTAPV1Witness { id: object::new(ctx) }); - b - }, - iv: version::v(1), - }; - - announce_interface_package( - default_tap.get_witness(), - vector[object::id(&default_tap)], - ); - - share_object(default_tap); -} - -/// with... -public struct DefaultTAPV1Witness has key, store { - id: UID, -} -``` - -### Worksheet - -The worksheet function is a core requirement of the Nexus Interface V1 specification. It creates a proof of UID (Unique Identifier) that serves as a "stamp collector" for tracking workflow execution state. This proof: - -1. Acts as a hot-potato object that collects execution confirmations from Nexus components like the DAG. -1. Must be constructed with a type defined in the same package and module that implements the interface. -1. Is used to verify that required operations have been performed in the correct sequence. - -```rust -use nexus_interface::version::InterfaceVersion; -use nexus_primitives::proof_of_uid::{Self, ProofOfUID}; - -public fun worksheet(self: &DefaultTAP): ProofOfUID { - self.iv.expect_v(1); - proof_of_uid::new_with_type(&self.get_witness().id, self.get_witness()) -} -``` - -### Tool Evaluation Confirmation - -The `confirm_tool_eval_for_walk` function is another core requirement of the Nexus Interface V1 specification. It is invoked by the Nexus Leader after a workflow contract has advanced the DAG to: - -1. Consume the worksheet hot-potato. -1. Verify that all required confirmations have been collected. -1. Complete the tool evaluation cycle for a specific walk in the workflow. - -```rust -use nexus_interface::version::InterfaceVersion - -public fun confirm_tool_eval_for_walk( - self: &mut DefaultTAP, - worksheet: ProofOfUID, -) { - self.iv.expect_v(1); - worksheet.consume(&self.get_witness().id); -} - -/// with... -fun get_witness(self: &DefaultTAP): &DefaultTAPV1Witness { - self.witness.borrow(b"witness") -} -``` - -## DAG Execution - -The Default TAP works in conjunction with the Nexus workflow engine, which provides: - -1. **DAG Implementation** - - Directed Acyclic Graph data structure for modeling complex workflows. - - Support for vertices, edges, and input/output ports. - - Entry group management for workflow initiation. - -1. **Tool Registry** - - Registration and management of available tools. - -1. **Tool Invocation** - - Support for tool execution invocation, onchain or offchain. - - Leader capability management. - -Where the above steps effectively ensure interface compliance, the (default) TAP also needs to enable DAG execution. - -### Begin DAG Execution - -The DAG execution is initiated through the `begin_dag_execution` function. - -This function: - -1. Takes a DAG and entry vertices with their input data. -1. Creates a worksheet to track execution state. -1. Begins execution of the entry group through the DAG. -1. Requests the network to execute walks through the DAG. -1. Shares the execution object for tracking progress. - -```rust -use nexus_primitives::data::NexusData; -use nexus_primitives::proof_of_uid::{Self, ProofOfUID}; -use nexus_workflow::dag::{DAG, Vertex, InputPort, EntryGroup}; - -/// Invokes the provided entry vertex on a DAG with the provided input data for -/// each input port. -public fun begin_dag_execution( - self: &mut DefaultTAP, - dag: &DAG, - network: ID, - entry_group: EntryGroup, - mut with_vertex_inputs: VecMap>, - ctx: &mut TxContext, -) { - self.iv.expect_v(1); - - let mut entry_vertices = vector[]; - let mut with_vertices_inputs = vector[]; - while (!with_vertex_inputs.is_empty()) { - let (vertex, inputs) = with_vertex_inputs.pop(); - entry_vertices.push_back(vertex); - with_vertices_inputs.push_back(inputs); - }; - - let mut worksheet = self.worksheet(); - - let (mut execution, ticket) = dag.begin_execution_of_entry_group( - &mut worksheet, - network, - entry_group, - entry_vertices, - with_vertices_inputs, - ctx - ); - - worksheet.consume(&self.get_witness().id); - - dag.request_network_to_execute_walks(&mut execution, ticket, ctx); - - public_share_object(execution); -} -``` - -## Security Considerations - -- The TAP uses witness tokens for authorization as required by the [Nexus Interface][nexus-interface]. -- Interface version checking ensures compatibility. -- Worksheet proofs ensure state integrity. -- Tool execution is properly isolated. - -For a broader security analysis of Nexus, refer to the [whitepaper section 4.4][whitepaper]. - -## Full Module Code - -Toggle to see the full module code for the default TAP: - -
- -Toggle code - -```rust -module nexus_workflow::default_tap; - -//! Module defining a default talus agent package with no added logic but -//! executing the provided DAG. This can be used for examples, tests or to run -//! JSON-defined DAGs from the CLI or via other means. - -use nexus_interface::version::InterfaceVersion; -use nexus_primitives::data::NexusData; -use nexus_primitives::proof_of_uid::{Self, ProofOfUID}; -use nexus_workflow::dag::{DAG, Vertex, InputPort, EntryGroup}; - -use sui::bag::{Self, Bag}; -use sui::transfer::{share_object, public_share_object}; -use sui::vec_map::{VecMap}; - -/// Shared object. -public struct DefaultTAP has key { - id: UID, - /// On package upgrade, we'll want to replace the previous witness with a - /// new one that identifies the new package. - /// - /// This is because as of now Sui doesn't give us access to the package ID - /// so we need to always create a new type and query it from the type via - /// its type name. - witness: Bag, - /// Clients can search the talus agent shared object for this type to - /// determine what interface to expect. - iv: InterfaceVersion, -} - -/// Used to identify the package::module deployment. -/// -/// Will also serve as authorization token to change list of shared objects for -/// nexus interface v1. -public struct DefaultTAPV1Witness has key, store { - id: UID, -} - -public(package) fun new(ctx: &mut TxContext) { - let default_tap = DefaultTAP { - id: object::new(ctx), - witness: { - let mut b = bag::new(ctx); - b.add(b"witness", DefaultTAPV1Witness { id: object::new(ctx) }); - b - }, - iv: nexus_interface::version::v(1), - }; - - nexus_interface::v1::announce_interface_package( - default_tap.get_witness(), - vector[object::id(&default_tap)], - ); - - share_object(default_tap); -} - -// === Client entry === - -/// Invokes the provided entry vertex on a DAG with the provided input data for -/// each input port. -public fun begin_dag_execution( - self: &mut DefaultTAP, - dag: &DAG, - network: ID, - entry_group: EntryGroup, - mut with_vertex_inputs: VecMap>, - ctx: &mut TxContext, -) { - self.iv.expect_v(1); - - let mut entry_vertices = vector[]; - let mut with_vertices_inputs = vector[]; - while (!with_vertex_inputs.is_empty()) { - let (vertex, inputs) = with_vertex_inputs.pop(); - entry_vertices.push_back(vertex); - with_vertices_inputs.push_back(inputs); - }; - - let mut worksheet = self.worksheet(); - - let (mut execution, ticket) = dag.begin_execution_of_entry_group( - &mut worksheet, - network, - entry_group, - entry_vertices, - with_vertices_inputs, - ctx - ); - - worksheet.consume(&self.get_witness().id); - - dag.request_network_to_execute_walks(&mut execution, ticket, ctx); - - public_share_object(execution); -} - -// === Nexus interface v1 === - -/// Nexus resources can prove that they did a thing. -public fun worksheet(self: &DefaultTAP): ProofOfUID { - self.iv.expect_v(1); - - proof_of_uid::new_with_type(&self.get_witness().id, self.get_witness()) -} - -/// One calls this after workflow contract is done with advancing the DAG. -public fun confirm_tool_eval_for_walk( - self: &mut DefaultTAP, - worksheet: ProofOfUID, -) { - self.iv.expect_v(1); - - worksheet.consume(&self.get_witness().id); -} - -fun get_witness(self: &DefaultTAP): &DefaultTAPV1Witness { - self.witness.borrow(b"witness") -} - -``` - -
- - - -[whitepaper]: https://talus.network/nexus/whitepaper.pdf -[nexus-interface-v1]: ../packages/nexus-interface.md#v1 -[nexus-interface]: ../packages/nexus-interface.md -[primitives]: ../packages/primitives.md +Historical data that was emitted by older witness TAP deployments may still be +decoded by compatibility event parsers, but it is not an executable runtime +path for new leader, SDK, CLI, or deployment flows. diff --git a/nexus/adr/address-balance-gas-payment.md b/nexus/adr/address-balance-gas-payment.md new file mode 100644 index 0000000..1898c76 --- /dev/null +++ b/nexus/adr/address-balance-gas-payment.md @@ -0,0 +1,223 @@ +# ADR-4: Address-Balance Gas Payments (SIP-58) — Removing the Leader Gas Pool + +**Status:** Proposed +**Date:** 2026-06-15 +**Authors:** Pavel ([@kouks](https://github.com/kouks)) +**Deciders:** Pavel ([@kouks](https://github.com/kouks)) +**Consulted:** Engineering Team +**Informed:** Engineering Team + +**Constraint Tags:** Technical | Architectural + +**Supersedes:** [ADR-2: Multi-Coin Gas Payment for the Leader Gas Pool](multi-coin-gas-payment.md) + +--- + +## Context + +### What problem are we solving? + +The leader pays for Sui transactions from a pool of `Coin` gas objects it +owns. Sustaining that pool requires a large, stateful subsystem: + +- discovery and seeding of owned coins at startup + (`be/leader/src/onchain/gas_pool.rs`); +- per-transaction acquisition of enough coins to cover a dry-run budget, plus + release/quarantine on failure (`be/leader/src/merchant/pipeline.rs`); +- settlement of survivor coins from transaction effects and dropping of smashed + coins (`be/leader/src/onchain/gas_pool.rs`); +- a Redis source of truth for coin metadata + (`be/leader/src/store/redis/redis_gas_manager.rs`); +- a consolidator loop that merges small claimed coins into reusable ones + (`be/leader/src/merchant/consolidator.rs`). + +The churn is driven by how the leader is reimbursed. On every verified walk +submission, the user's execution payment reimburses the leader's gas by +**minting a brand-new `Coin`** to the leader and transferring it +(`consume_payment_for_verified_leader_submission` → +`settle_nonzero_balance` in `sui/interface/sources/tap.move:1295-1308, 3840-3846`, +which does `public_transfer(coin::from_balance(balance, ctx), leader)`). The pool +then has to re-discover and consolidate those coins to keep them usable. ADR-2 +made this workable by paying each transaction with _multiple_ coins and tracking +metadata in Redis, but it kept the entire pool/consolidator/quarantine machinery +in place. + +Sui's **address balances** (SIP-58) let a transaction pay gas directly from the +sender's `SUI` address balance, with no gas coin objects at all. If the leader +pays gas from its address balance — and is reimbursed _into_ that balance instead +of into fresh coins — the whole gas-coin subsystem becomes unnecessary. + +### Relevant Constraints + +**Technical:** + +- The pinned `sui-transaction-builder` rejects gas-less transactions: + `TransactionBuilder::try_build()` returns `Err(MissingGasObjects)` when no gas + object is set (`builder.rs:322`). However, the `sui_sdk_types::Transaction` it + returns exposes a **public** `gas_payment: GasPayment` whose + `objects: Vec` field is public + (`transaction/mod.rs:36-40, 124-138`). We can therefore finish the build with a + throwaway placeholder gas object and then clear `gas_payment.objects` on the + returned transaction — no fork or SDK change required. +- SIP-58 requires a transaction paying gas from an address balance to set + `expiration = TransactionExpiration::ValidDuring { min_epoch, max_epoch, +chain, nonce, .. }` with `min_epoch == max_epoch ==` the current epoch + (`transaction/mod.rs:90-103`). The leader must fetch the current epoch and the + chain identifier at submission time and supply a nonce for replay prevention. +- The framework primitive `balance::send_funds(balance, recipient)` deposits + a `Balance` directly into the recipient's address balance. + +**Architectural:** The transaction's gas budget is only known after a dry run. +The merchant already dry-runs before submission, so the budget is available +before the transaction is finalized. + +### Architectural Position + +This decision affects the **leader gas pool**, the **merchant submission +pipeline**, and a single **on-chain settlement helper**. Rather than maintaining +a pool of gas coins, the leader pays gas from its address balance and is +reimbursed into that same balance. + +--- + +## Decision + +### What are we doing? + +We pay transaction gas from the leader's **Sui address balance** and delete the +gas-coin subsystem. + +1. **Address-balance gas payment.** The merchant builds each PTB as today, then + finishes it through a leader-local extension trait + (`FinishWithAddressBalanceGas` on `sui::tx::TransactionBuilder`): it feeds the + builder one placeholder gas object to satisfy the builder's non-empty check, + calls `finish()`, clears `gas_payment.objects` on the resulting transaction, + and sets a `ValidDuring` expiration pinned to the current epoch with a chain + digest and a retry-distinct nonce. + +1. **Remove the gas pool.** The gas pool, the gas-manager coin methods and Redis + coin keys, the consolidator process, the quarantine mechanism, and + effects-based coin settlement are deleted. Payment-lock storage (per + `(execution, vertex)`) is unrelated to gas coins and is retained. + +1. **Reimburse into the address balance.** Only the leader-reimbursement path + changes on-chain: `consume_payment_for_verified_leader_submission` settles the + charged balance via `balance::send_funds` into the leader's address + balance instead of minting and transferring a new coin. Payer/user refunds and + tool-owner/collateral claims continue to materialize coins. + +1. **Funding and observability.** The operator funds the leader's address balance + out-of-band. The leader adds a read-only startup check that warns or fails + below a configurable floor (`SUI_MIN_ADDRESS_BALANCE_MIST`) and a periodic task + that exports the current address balance as a gauge + (`leader_address_balance_mist`). Auto-converting owned coins into the address + balance at startup is left as a documented TODO. + +### Why this approach? + +Address-balance gas removes the root cause of the pool's existence: there are no +gas coins to size, select, smash, settle, consolidate, or quarantine, and no +per-transaction coin-count ceiling. The builder limitation is worked around with +a few lines against public types, so no SDK fork or multi-repo lockstep is +needed. Scoping the on-chain change to the leader-reimbursement path keeps the +blast radius minimal and avoids changing behavior for external payers and tool +owners, who may not be able to redeem funds from an address balance. + +--- + +## Alternatives Considered + +### Alternative 1: Keep the multi-coin gas pool (ADR-2) + +**Description:** Retain the pool, paying each transaction with as many coins as +needed and tracking metadata in Redis. + +**Why not:** ADR-2 is a sophisticated solution to a problem that address balances +remove entirely. It keeps a large stateful subsystem (pool, consolidator, +quarantine, effects settlement, Redis schema) with its own failure modes — +balance drift, equivocation windows on re-add, the 256-coin per-transaction +limit. Address balances make all of it unnecessary. + +### Alternative 2: Put the builder bypass in the SDK + +**Description:** Add the address-balance finish helper to the sibling `nexus-sdk` +`sui` wrapper so it is reusable across repos. + +**Why not:** The bypass uses only public `sui_sdk_types` fields and compiles +locally in the leader crate. Putting it in the SDK would require the SDK-spanning +workflow (path-link, push a branch, switch back) for a change that does not need +it. It can be promoted to the SDK later if another consumer appears. + +### Alternative 3: Route all settlement paths to address balances + +**Description:** Change `settle_nonzero_balance` in place so refunds, +tool-owner claims, and collateral also deposit into address balances. + +**Why not:** Recipients of address-balance funds must be able to redeem them or +pay gas from them. The leader gains that capability; arbitrary payers, users, and +tool owners may not, and silently moving their funds out of coin objects is a UX +regression and an SDK-compatibility risk for external integrators. We add a +second helper and switch only the leader path, leaving the rest byte-identical. + +### Alternative 4: Auto-convert owned coins into the address balance at startup + +**Description:** On boot, deposit the leader's owned `Coin` into its address +balance via `send_funds` so the leader is self-funding. + +**Why not (now):** It depends on the framework `send_funds` entry being exposed +through the SDK idents, which may require an SDK change. External funding plus a +startup balance check is sufficient for the first cut; auto-conversion is recorded +as a follow-up TODO. + +--- + +## Consequences + +### Positive Consequences + +- A large, stateful, failure-prone subsystem (pool, manager coin state, + consolidator, quarantine, effects settlement) is deleted. +- No per-transaction gas-coin ceiling and no coin-distribution drift; gas + payment is stateless and needs no current object references. +- The on-chain change is a single, surgical settlement-sink switch with no impact + on payer/user refunds or tool-owner claims. + +### Negative Consequences + +- The gas payment relies on a builder bypass (clearing `gas_payment.objects` on + the returned transaction); a future `sui-transaction-builder` change to those + public fields could require revisiting it. +- The network must have the address-balance / funds-accumulator protocol feature + enabled, and the leader's address balance must be funded out-of-band; an empty + balance halts submissions. +- The leader must fetch the current epoch and chain identifier per submission and + derive a nonce, adding a small RPC dependency to the hot path (epoch can no + longer be assumed static across the process lifetime). + +### Neutral Consequences + +- Reimbursements no longer appear as coins in the leader's wallet; they accrue in + the address balance, observed via the new gauge. + +### Reversibility Assessment + +- **Reversibility:** Medium +- **Reversal cost:** The offchain change is internal to the leader and mechanical + to revert. The on-chain settlement switch is the stickier part: reverting it + after funds have accrued in address balances requires a deliberate migration + back to coin materialization. +- **Point of no return:** None in code; operationally, once reimbursements flow + to the address balance, reverting requires draining/redeeming that balance. + +--- + +## Context Evolution Tracking + +### Review Schedule + +- **Next review:** After e2e confirms address-balance gas on localnet and after + observing the leader's address-balance drawdown over a representative period in + a real deployment. +- **Review criteria:** The builder bypass holding across SDK bumps; address- + balance protocol availability on target networks; whether external funding plus + the startup check is operationally sufficient or auto-conversion is warranted. diff --git a/nexus/adr/auto-abort-expired-executions.md b/nexus/adr/auto-abort-expired-executions.md new file mode 100644 index 0000000..ade5d79 --- /dev/null +++ b/nexus/adr/auto-abort-expired-executions.md @@ -0,0 +1,414 @@ +# ADR-5: Auto-Abort of Expired DAG-Walk Executions + +**Status:** Proposed +**Date:** 2026-06-25 +**Authors:** Pavel ([@kouks](https://github.com/kouks)) +**Deciders:** Pavel ([@kouks](https://github.com/kouks)) +**Consulted:** Engineering Team +**Informed:** Engineering Team + +**Constraint Tags:** Technical | Architectural | Operational + +--- + +## Context + +### What problem are we solving? + +A DAG execution runs as a set of concurrent **walks**. Each active walk pins a +`next_vertex` and a deadline: on-chain it is `created_at + timeout_ms * 2` +(`sui/workflow/sources/execution.move` → `DAGWalk::Active`, +`is_active_walk_expired`). Off-chain, the same walk is distributed to leaders +with a matching deadline budget: the `RequestWalkExecution` event carries +distribution metadata (`requested_at`, `deadline`, an ordered `leaders` vector), +and `Distribution::decide` (`be/leader/src/onchain/distribution.rs`) computes +`deadline_at = requested_at + deadline` (the primary's window) and +`double_deadline_at = deadline_at + deadline` (the backup's window). The latter +coincides with the on-chain `created_at + timeout_ms * 2` expiry. + +The on-chain contract already provides a permissionless way to clear an expired +walk — `execution_resolution::abort_expired_execution` and, when the expired +vertex still holds a locked TAP payment, `gas_extension::abort_expired_execution_with_tool_gas` +— and the nexus-sdk already exposes builders for both. Our e2e tests exercise +these by submitting the abort by hand. + +**No production leader code ever calls them.** When a walk passes +`double_deadline_at` without being evaluated, `Distribution::decide` returns +`Ignore` and the leader simply forgets the walk. Nothing on chain advances: + +- the walk stays `Active` forever, +- the vertex's TAP payment stays **locked** (the invoker's funds are stuck), +- `ExecutionFinished` is never emitted, so the execution never settles or + refunds. + +### Impact + +Every walk that a primary fails to evaluate and a backup does not pick up leaves +a permanently stalled execution with funds locked on chain. This is silent — no +leader is responsible for the cleanup — and only resolves through manual, +out-of-band abort submission. As walk timeouts are routine (slow tools, leader +restarts, transient failures), this is a recurring source of stuck executions +and trapped invoker funds rather than a rare edge case. + +### Relevant Constraints + +**Technical:** The abort entries are **permissionless** — they take the +`leader_registry` for bookkeeping but require no leader capability, and the Move +code re-checks `is_active_walk_expired` itself, so an abort against an +already-resolved or not-yet-expired walk is a safe no-op/rejection rather than a +corruption risk. The leader already has, for every walk it handled, the data +needed to identify it: the `DAGExecution` id and the `walk_index`. It also +already tracks which vertices have locked payments (`save_gas_lock` on +`PaymentLockUpdate`), which is exactly the signal that decides plain-vs-tool-gas +abort. The leader pays its own gas via address-balance gas +(`finish_with_address_balance_gas`); an abort **refunds** the invoker, so there +is no execution payment to recoup the abort's gas from. + +**Architectural:** The leader's submission path is the merchant pipeline: every +outbound transaction is a `NexusTransactionKind` variant implementing +`TransactionLike` (build the PTB, declare capabilities, run a `preflight` that +may `Skip`, finish with gas). Transactions are delivered through a Redis-backed +**scheduled high-integrity channel** that (a) can defer a payload to a wall-clock +time (`ScheduleSpec::at`), (b) deduplicates by the payload's `Identifiable::id()`, +and (c) **survives process restart** — a scheduled item persists in Redis and +fires after a restart. The event listener already holds both the event sender +and this merchant sender. + +That channel, however, **bounds retries**: a transaction is attempted at most +`MERCHANT_MAX_RETRIES + 1` times (default 4) with only immediate / fixed-500ms +backoff, and on exhaustion it is **permanently deleted** from Redis — there is no +dead-letter queue and no retry-forever path (the drop is centralized in the +scheduler queue discipline, `be/leader/src/channel/scheduler/queue.rs`, in +`on_nack` and `on_enqueue`). During a multi-hour chain outage every submission +attempt fails transiently, so an abort would exhaust its budget within minutes +and be dropped — leaving the execution stalled exactly as before. The fix must +therefore make the abort **retry until it lands**, not just retry a few times. + +**Operational:** Leaders restart routinely (Cloud Run instance refreshes — see +[ADR-3](leader-status-ownership-versioning.md)). Any abort mechanism must keep +working across a restart without re-deriving in-flight state from scratch, and +must not depend on a clean shutdown. There is no on-chain index of "all active +executions assigned to me," so enumerating every expired walk from chain is not +cheaply available. + +### Architectural Position + +This decision adds an **abort responsibility** to the leader's existing walk +lifecycle. It spans the distribution decision (`onchain/distribution.rs`), the +event listener (`onchain/listener.rs`), and the merchant transaction set +(`merchant/transaction.rs`, a new `executor/handlers/abort_expired.rs`). It is +purely off-chain leader behavior — no Move or SDK change — because the on-chain +entries and SDK builders already exist. + +--- + +## Decision + +### What are we doing? + +For every walk a leader is responsible for, the leader **arms a deferred +abort-check transaction** at the walk's double-deadline, reusing the existing +Redis-backed scheduled merchant channel. When the abort-check fires, it re-reads +chain state and either aborts the still-expired walk or no-ops if the walk has +since resolved. + +1. **New transaction kind `AbortExpiredExecution`.** A `NexusTransactionKind` + variant (params in `executor/handlers/abort_expired.rs`, modeled on + `ExecuteScheduledOccurrenceTxParams`) carrying the `DAGExecution` id and + `walk_index`. Its `Identifiable::id()` is `abort-expired-{execution}-{walk_index}` + — distinct from event ids and execute transactions, and stable so the channel + deduplicates redundant arming. + +1. **Self-verifying preflight.** Before submission the merchant runs + `preflight`, which fetches the `DAGExecution`, indexes `walks[walk_index]`, + and calls `DagExecutionWalk::expired_active_vertex(now)` against a trusted + `CheckpointClock` snapshot. If the walk is no longer an expired active walk + (it advanced, failed, was aborted, cancelled, or is already pending-abort) the + transaction returns `PreflightOutcome::Skip` and never hits the chain. This is + how "if the walk resolved, drop it" is realized — no explicit cancellation is + needed. + +1. **Plain-vs-tool-gas chosen at fire time.** When the expired vertex still holds + a locked payment (per the gas-lock store), `preflight` resolves the relevant + `ToolGas` object references and `into_sui_tx_data` emits one + `abort_expired_execution_with_tool_gas` call per `ToolGas` (unlocking/refunding + the vertex payment before recording abort intent); otherwise it emits the + plain `abort_expired_execution`. Resolving this at fire time — not at arm time + — keeps the decision correct even as lock state changes during the window. + +1. **Leader sponsors the abort gas.** `is_sponsored()` returns `true`: the abort + refunds the invoker, so there is no execution payment to recoup from, and the + leader eats the gas (like `ActivateLeader`/`RequestCapability`). + +1. **Arming, at decision time, in the listener.** `Distribution` gains a sibling + helper `abort_schedule(event, claimed_token)` (leaving `decide` and + `HandlingDecision` untouched). It returns an abort-check to arm **iff** the + event is an abortable walk event (`RequestWalkExecution`, or the inner request + of `RequestScheduledWalk`), this instance is the current claim owner + ([ADR-3](leader-status-ownership-versioning.md)), and this instance's leader + cap is in the event's `leaders` vector. The fire time is + `double_deadline_at` plus a position-dependent grace stagger (see item 6 — + the primary fires at `+auto_abort_grace_period`). The listener schedules the + resulting `AbortExpiredExecution` onto the merchant channel via + `ScheduleSpec::at(...)`, behind a config toggle. + +1. **Any tracking leader may abort, staggered by position.** Both the primary + (`leaders[0]`) and backup (`leaders[1]`) arm the abort, but the fire time is + staggered by the leader's position in the distribution vector: the primary + fires at `double_deadline_at + grace`, the backup at `double_deadline_at + + 2*grace` (position `n` at `+ (n+1)*grace`). Staggering means the primary + normally lands the abort first and the backup's copy self-skips, so the two + do not submit competing transactions against the same shared execution object + at the same instant (which would otherwise cause transient version-conflict + errors). The Move call is permissionless and self-rejects once the walk is no + longer active+expired, so the backup's copy is harmless whether it skips or + lands. This still maximizes liveness: if the primary is down, the backup + aborts one `grace` window later. + +1. **Durable retry through chain outages.** The merchant channel normally drops a + transaction after `MERCHANT_MAX_RETRIES` attempts; for the abort that would + lose the cleanup during a long outage. We add an **opt-in unbounded-retry + policy to the scheduler core**: a payload may declare + `RetryPolicy::Unbounded { base_ms, cap_ms }` (via a new `Schedulable::retry_policy()`, + default `Bounded`), carried in the `SchedulerHint` and persisted on the + `SchedulerMetadata`. For such a message, the queue discipline + (`queue.rs::on_nack`/`on_enqueue`) **never deletes on exhaustion** and instead + re-schedules it with **capped exponential backoff** + (`min(cap_ms, base_ms * 2^attempts)`, defaults base 1s / cap 5min, + configurable). The next-ready time is persisted in Redis, so the backoff + survives restart. The `AbortExpiredExecution` tx opts into this policy; all + other transactions are unchanged (`Bounded`). The retry loop terminates by + construction: once the walk resolves, `preflight` returns `Skip` (which + **acks** and removes the message); a deterministic on-chain failure is `Fatal` + (also acks); only transient/chain-down failures back off and retry. So an + abort retries every ≤5min until the chain recovers and the abort lands or the + walk is otherwise resolved. + + For this to hold, the merchant pipeline must classify an **unreachable Sui** + as a transient (retryable) failure, never `Fatal` — a `Fatal` acks and drops + the message. The pipeline already does this: every RPC-connectivity boundary + (gRPC client construction, submission-context fetch, dry-run simulation, and + the live execute-and-wait) returns a transient error, while the only `Fatal` + paths are deterministic (transaction-data conversion errors and dry-run + rejections the chain actually executed and returned). We keep this invariant: + the abort's `preflight` (and its preflight-time ref resolution) surfaces RPC + failures as transient errors and never as `Skip`, and `into_sui_tx_data` does + no network I/O. + +1. **Best-effort restart coverage.** Scheduled abort-checks persist in Redis and + fire after a restart, so a leader that armed an abort before crashing still + aborts after coming back. Replayed terminal events keep the rest of the store + consistent. We do **not** enumerate chain state to abort walks the leader + never armed. + +### Why this approach? + +The scheduled channel already solves the hard parts of this problem — durable +deferral to a wall-clock deadline, deduplication, and restart survival — so the +abort rides it as just another transaction kind. The `preflight`/`Skip` +machinery already exists for exactly this shape (see +`ExecuteScheduledOccurrenceTxParams`, which skips stale scheduled occurrences), +so "abort only if still expired" needs no new control flow and no per-walk +bookkeeping to tear down. Keeping `decide` unchanged and adding a parallel +`abort_schedule` avoids forking the decision logic across its two call sites +(the listener and `decide_task_handling`), which is where subtle distribution +bugs hide. The result is additive: no Move change, no SDK change, no new +long-running task, and a single feature flag to disable it. + +--- + +## Alternatives Considered + +### Alternative 1: Background reaper over the persistent task store + +**Description:** Persist every owned walk in the Redis task store, and run a +periodic background task that scans the store, computes each walk's +double-deadline, fetches the `DAGExecution` for those past it, and submits an +abort for any still expired. Resolution is handled by the existing terminal-event +`drop_walk_task` cleanup; restart re-enumerates the persistent store. + +**Why not:** It is a strictly larger surface for the same outcome. It introduces +a new always-on loop, broadens task-store writes to every walk (today only the +single-leader-primary and backup branches persist), adds a `list_tasks` +enumeration method, and pays a periodic RPC cost to poll chain state. The +scheduler-reuse approach gets durable timing and restart survival from +infrastructure we already run, with no polling. The reaper's one genuine +advantage — re-deriving tracking purely from the store on restart — is matched +closely enough by the scheduled channel's own Redis persistence. Worth +reconsidering if we later need coverage of walks the leader never armed (see +Alternative 3). + +### Alternative 2: New `HandlingDecision::AbortExpired` variant threaded through `decide` + +**Description:** Rather than a sibling helper, extend `HandlingDecision` with +abort-bearing variants (e.g. `ExecuteNowThenAbort`, `WaitThenAbort`, +`AbortExpired`) and rewrite the `decide` match arms so the doubly-expired +branches produce an abort instead of `Ignore`. + +**Why not:** `decide` is invoked from two places (the listener and the consumer's +`decide_task_handling`) that must agree; the consumer path would need new arms to +map the abort variants back to execute/ignore, and any drift between the two call +sites silently breaks distribution. A second scheduled **event** delivery is also +awkward because the channel deduplicates by event id and would collide with the +in-flight execute delivery. Producing a distinct **transaction** (different id, +merchant channel) and computing it via a side helper sidesteps both problems. + +### Alternative 3: Full on-chain enumeration of assigned expired walks + +**Description:** On startup (and/or periodically) enumerate all in-flight +executions assigned to this leader from chain and abort any expired walk — +including walks this leader never observed (e.g. created before the feature +shipped, or during an outage). + +**Why not:** There is no cheap on-chain index of active executions assigned to a +leader, so this requires significant query machinery and ongoing RPC cost for a +small tail. We explicitly accept best-effort coverage for now. This remains the +upgrade path if the uncovered tail proves material in practice. + +### Alternative 4: Restrict abort to the primary, with a backup fallback window + +**Description:** Only `leaders[0]` arms the abort at the double-deadline; the +backup waits a further grace window (e.g. a triple-deadline) before attempting, +mirroring the primary/backup execution handoff. + +**Why not (and what we shipped):** A full primary-only handoff with a large +extra fallback window adds windowing complexity and latency before a down +primary's walks get cleaned up. Instead we ship a lightweight middle ground: +both leaders arm, but staggered by one `grace` window per position (primary at +`+grace`, backup at `+2*grace`). The stagger gives the primary first attempt — +so the backup's permissionless copy almost always self-skips rather than racing +the primary on the shared execution object — while still falling back to the +backup just one `grace` window later if the primary never lands it. This keeps +liveness without the same-instant version-conflict races that arming both at the +identical time would cause. + +### Alternative 5: Bounded retry tweaks instead of an unbounded policy + +**Description:** To survive outages, either (a) raise `MERCHANT_MAX_RETRIES` to a +large value, or (b) keep the scheduler core untouched and have the abort tx +re-arm itself on transient failure by scheduling a fresh delivery (a distinct +"epoch" id) at `now + backoff` before it can be dropped. + +**Why not:** (a) is global (affects every transaction class), still finite, and +hammers the RPC with the current near-zero backoff — not "until success." (b) +works but builds bespoke retry bookkeeping outside the scheduler (epoch ids, +ack-then-reschedule crash windows, its own backoff state) and is not reusable. +Adding a first-class `RetryPolicy::Unbounded` to the scheduler keeps one message +identity, reuses the existing Redis-persisted ready-time/backoff machinery, has a +clean restart story, and is opt-in so bounded transactions are unaffected — and +it becomes a capability any future durable transaction can use. + +### Alternative 6: Do nothing (manual abort) + +**Description:** Keep clearing stalled expired executions by submitting aborts +out of band. + +**Why not:** Walk timeouts are routine, the failure is silent, and it traps +invoker funds until a human intervenes. It does not self-correct. + +--- + +## Consequences + +### Positive Consequences + +- Expired walks are cleared automatically: the execution settles/refunds, locked + invoker funds are released, and `ExecutionFinished` is emitted instead of the + execution stalling forever. +- No on-chain or SDK change — the feature is entirely additive leader behavior + over existing Move entries, SDK builders, and channel infrastructure. +- Restart-resilient by construction: armed aborts persist in Redis and fire after + a restart, with no dependence on a clean shutdown. +- Survives prolonged chain outages: the abort retries with capped exponential + backoff until it lands (or the walk resolves), instead of being dropped after + the merchant's bounded retries. The new `RetryPolicy::Unbounded` is reusable by + any future durable transaction. +- Liveness under leader failure: a down primary no longer blocks cleanup because + the backup arms the same abort. +- Disable-able via a single config toggle for safe rollout. + +### Negative Consequences + +- An abort-check is armed for **every** owned walk, including walks that complete + normally; those fire at the double-deadline and incur a chain read in + `preflight` before Skipping. This is wasted work proportional to throughput + (mitigated by the dedup id and the Skip-without-submission path; a future + optimization could cancel armed aborts on terminal events if the channel grows + a cancel API). +- Walks of the same execution each arm their own abort; the first to land aborts + all currently-expired walks, and later firings no-op via Skip — bounded + redundancy, but redundancy nonetheless. +- Distinct leaders (primary and backup, separate Redis) both arm an abort, but + staggered by one `grace` window per position so the primary fires first and the + backup's copy usually self-skips. In the rare overlap (primary lands within the + stagger window) the backup's copy is a rejected/Skipped no-op — at most a little + extra gas on the loser, not a same-instant version-conflict race. +- The unbounded-retry policy modifies the shared scheduler core + (`channel/scheduler/`). The change is guarded so bounded transactions keep + their exact current drop-on-exhaustion behavior, but it widens the blast radius + of this work beyond the abort path and needs its own focused tests. +- An `Unbounded` message never self-expires: it stops only on `Skip`/success/ + `Fatal`. For the abort this is the intended behavior (resolution always + produces a `Skip`), but it means a bug that never produces a terminal outcome + would retry indefinitely — bounded only by the ≤5min cap, and observable via + the backoff metrics/logs. +- **Gas-griefing vector (accepted risk).** The abort is permissionless and + **leader-sponsored** (`is_sponsored() == true`): the leader pays the abort gas + from its own address balance while the invoker's execution payment is fully + refunded. There is **no per-invoker or aggregate cap** on how many aborts a + leader will arm/fire. An actor can therefore create executions and deliberately + let their walks expire, draining a small amount of leader gas per abort while + getting refunded. We accept this for now: the alternative — leaving expirations + unaborted — is the status-quo failure this ADR exists to fix (stalled + executions, indefinitely locked invoker funds), and refusing the abort would + not recover the gas anyway. The per-abort gas cost is small, and abuse is + observable: the `leader_auto_abort_armed_total` counter (and the + `leader_auto_abort_outcomes_total` breakdown) make an anomalous arming rate + visible. **Mitigation: alert on the armed-total rate** (see Review criteria); if + abuse becomes material, follow-ups include rate-limiting arming per invoker, + charging abort gas against the refund, or an on-chain expiry deposit — none of + which are needed for the initial rollout. + +### Neutral Consequences + +- Best-effort coverage only: walks armed before the feature shipped, or lost to a + crash between `abort_schedule` and the `schedule` call, are not aborted. Accepted + (Alternative 3 is the upgrade path). +- The abort is sponsored by the leader (`is_sponsored() == true`); abort gas is + not recouped, consistent with refunding the invoker. +- `decide` and `HandlingDecision` are unchanged; the abort is an independent, + parallel concern. + +### Reversibility Assessment + +- **Reversibility:** High +- **Reversal cost:** Set `EXECUTOR_AUTO_ABORT_EXPIRED_ENABLED=false` to disable + at runtime; fully reverting removes a transaction kind and a listener call with + no on-chain or migration impact. +- **Point of no return:** None — no schema, contract, or persisted-format change. + +--- + +## Context Evolution Tracking + +### Review Schedule + +- **Next review:** After observing auto-abort behavior on testnet across normal + walk timeouts and at least one leader rolling restart. +- **Review criteria:** Whether expired executions reliably clear and funds + unlock; the volume of `preflight`-Skipped abort-checks (cost of arming for + every walk) and whether a cancel-on-terminal optimization is warranted; the + rate of redundant cross-leader aborts; whether the best-effort coverage gap + bites in practice (motivating Alternative 3); whether `auto_abort_grace_period` + and the retry base/cap backoff are tuned correctly; whether the unbounded-retry + policy behaves well in a real outage (aborts land promptly on recovery, no + runaway retry loops, bounded transactions unaffected). +- **Required alert:** wire an alert on the **rate** of + `leader_auto_abort_armed_total` (and watch `leader_auto_abort_outcomes_total`): + a sustained spike signals either a systemic execution failure or the + gas-griefing vector above. Alerts live outside this repo (the leader only + exposes `/metrics`); this is an operational follow-up to configure where the + fleet's Prometheus rules are managed. Revisit whether a per-invoker arming cap + or charging abort gas to the refund is warranted based on what the alert shows. + +--- diff --git a/nexus/adr/leader-status-ownership-versioning.md b/nexus/adr/leader-status-ownership-versioning.md new file mode 100644 index 0000000..a2372f5 --- /dev/null +++ b/nexus/adr/leader-status-ownership-versioning.md @@ -0,0 +1,335 @@ +# ADR-3: Claim-Token-Guarded Leader Activation and Suspension + +**Status:** Proposed +**Date:** 2026-06-10 +**Authors:** Pavel ([@kouks](https://github.com/kouks)) +**Deciders:** Pavel ([@kouks](https://github.com/kouks)) +**Consulted:** Engineering Team +**Informed:** Engineering Team + +**Constraint Tags:** Technical | Architectural | Operational + +--- + +## Context + +### What problem are we solving? + +A leader's on-chain liveness is a single `LeaderStatus` (`Active` / `Suspended` / +`Slashed`) field on its per-leader record in `nexus_registry::leader` +(`sui/registry/sources/leader.move`). Only `Active` leaders are selected into the +distribution committee. The leader process sets itself `Active` on startup once +its event listener has caught up to the live checkpoint +(`be/leader/src/onchain/listener.rs` → `ensure_leader_status_active`), and sets +itself `Suspended` on shutdown in the SIGTERM handler +(`be/leader/src/services.rs` → `set_leader_inactive`). + +The shutdown handler suspends **unconditionally**. It does not check whether this +instance is the one that owns the current `Active` state, or whether another +instance is still alive serving as the leader. During a Cloud Run rolling restart +— which briefly runs two instances of the same leader before consolidating back to +one — the shutdown of _any_ instance flips the shared record to `Suspended`, +removing the leader from the distribution vector even though a healthy instance is +still running. + +This is independent of, and additional to, the activation-on-catchup feature +(shipped 2026-04-23): that only protects the **activation** side of the race, not +the **deactivation** side. + +### Production impact + +Observed on testnet leaders: + +- **leader-0 (us-east1):** `Suspended` for at least 36 days, recovered only by a + manual `scale 0→1`. Zero work served as primary or backup over the window. +- **leader-2 (us-west1):** `Suspended` for ~23.5 hours after a Cloud Run instance + refresh on 2026-05-28 (no deploy that day). Reproduced from logs: instance A + starts and catches up, the old instance suspends correctly, instance B starts, + A activates correctly, B no-ops, then Cloud Run kills one of the two instances + and the unconditional suspend flips the record to `Suspended` — with no further + status change until manual intervention. +- **leader-1 (eu-west1)** carried the entire testnet workload alone throughout — + a single point of failure with no backup leader for any execution. + +This recurs on **every** Cloud Run instance refresh (zone maintenance, instance +auto-refresh, hardware recovery), with unpredictable timing. + +### Relevant Constraints + +**Technical:** `status` is one field on **one** shared record per leader, not +per-instance. Any instance holding a valid leader capability can write it. To +distinguish "the instance that owns the current `Active` state" from "any other +instance," we need a per-activation **ownership token** that is unique across +instances with no inter-instance coordination — uniqueness, not ordering, is what +the problem requires. Sui already gives us exactly that: every transaction has a +globally unique digest (the protocol forbids digest replay), readable on-chain via +`tx_context::digest(ctx)` (`public fun digest(&TxContext): &vector`, already +used in `sui/workflow/sources/dag.move` and `sui/scheduler/sources/scheduler.move`). +Transactions on the shared registry object are serialized by consensus, giving a +well-defined last writer. The shutdown suspend TX runs inside a bounded +shutdown-timeout budget (`timeout_tx`, 2/5 of the configured shutdown timeout), so +it must not abort or retry on the expected "I am not the owner" path. + +**Architectural:** Move functions that off-chain code calls are exported through +the nexus-sdk ident registry (`sdk/src/idents/workflow.rs`), so any new Move entry +point requires an SDK change landed before the leader can call it. This ships as +part of a new framework version that redeploys the registry package and +re-registers leaders, so changing the on-chain `Leader` struct layout is free. + +**Operational:** The orchestrator (Cloud Run) may run two instances of the same +leader during a rolling restart and may terminate either one. A killed instance +cannot run its shutdown handler at all; a gracefully-stopped instance can. + +### Architectural Position + +This decision affects the **leader status lifecycle** spanning the Move +`nexus_registry::leader` module, the leader's activation path +(`onchain/listener.rs`, `onchain/leader_status.rs`) and suspension path +(`services.rs`). It introduces a notion of **ownership** of the active state, keyed +by a unique per-activation **claim token**, so that only the instance that +currently owns `Active` can suspend it. + +--- + +## Decision + +### What are we doing? + +We make activation **claim ownership** by writing a unique claim token, and make +suspension **conditional** on still holding that exact token. The token is the +digest of the activating transaction itself; ownership is decided by last writer +(Sui serializes writes to the shared registry). A periodic recheck heals the +record if it is left `Suspended` while a live instance remains. + +1. **On-chain ownership field.** Add `claim_token: vector` to the Move `Leader` + struct. `bootstrap_leader_record` initializes it to the bootstrap transaction's + own digest (`*ctx.digest()`) — a unique, non-empty starting value that no + instance holds, so it can never accidentally match a suspender's token. + +1. **Token = activating transaction digest.** On activation, Move stores + `claim_token = *ctx.digest()`. Sui guarantees this is globally unique (no digest + replay), so two instances activating concurrently always write distinct tokens — + there is no version to collide and no tiebreak to design. The activating + instance learns the digest of its own activation transaction from that + transaction's receipt and remembers it in process (`LeaderState.claimed_token`). + +1. **Two new Move entry points**, leaving `set_status` untouched for admin/CLI: + - `activate_and_claim(registry, leader_cap, clock, ctx)` — aborts only if the + leader is `Slashed`; sets `status = Active`; sets `claim_token = *ctx.digest()`; + emits a `LeaderClaimedEvent { claim_token }`. Takes no token argument; the + token is intrinsic to the transaction. + - `suspend_if_token(registry, leader_cap, token, clock, ctx) -> bool` — **never + aborts on mismatch**. No-ops (returns `false`) if `status != Active`; suspends + and returns `true` only if `claim_token == token`; otherwise emits a + `LeaderSuspensionSkippedEvent { attempted, owner }` and returns `false`. It + does not clear the token; the next activation overwrites it. + +1. **Capture at activation, replay at suspension.** The instance records the + activation transaction's digest (from the receipt of the activation it just + landed) into `LeaderState.claimed_token`, and the shutdown handler sends **that + stored token**. There is nothing to recompute at shutdown — the token is a fixed + value tied to a specific past activation, which is precisely why a departing + instance that no longer owns the record cannot match. + +1. **Activation always claims.** The existing "already `Active` → no-op early + return" in `ensure_leader_status_active` is removed, so a newer instance writes + its own token and takes ownership instead of passively remaining a non-owner. + +1. **Periodic recheck.** A live, caught-up, non-shutting-down instance re-reads its + status every `APP_LEADER_STATUS_RECHECK_INTERVAL` (default **60s**, env-config). + If it observes `Suspended`, it re-activates (writing a fresh token it then + stores) and reclaims ownership; if `Active`, it does nothing. + +### Task-acceptance gate + +The claim token is not only the suspend guard — it also decides _which instance +does the work_. Without this, two `Active` siblings of one leader (the rolling-restart +window) would both accept and execute the same distributed tasks, doubling on-chain +submissions and gas spend. + +1. **Publish the claim to Redis — immediately, then via the event.** When an + instance activates, it writes its own claim token into Redis right where it + captures the activation digest (post-receipt), so it can serve tasks at once + without waiting to index its own event. Separately, `activate_and_claim` emits + `LeaderClaimedEvent { registry, leader_cap_id, claim_token }`; the listener + delivers it like any other event and a consumer handler writes `claim_token` + into the leader's Redis namespace. The event is what tells a _sibling_ — an + already-running instance that did not just activate — that a newer instance has + claimed, so it stops accepting tasks. The handler filters by `leader_cap_id`, + so another leader's claim never clobbers this one. Because instances of the + same leader share one namespace, the latest claim is visible to every sibling. + +1. **Gate `decide` on ownership.** When `Distribution::decide` handles a + _distributed_ task, it compares this instance's in-process `claimed_token` + against the claim stored in Redis. Only the instance whose token matches acts + on the task; a non-owner returns `Ignore`. Non-distributed events are never + gated — they keep syncing on every instance (including `LeaderClaimedEvent` + itself). + +1. **Permissive fallback.** If no claim has been synced yet (the startup window), + the gate is permissive so single-instance behavior is never regressed. A Redis + failure surfaces as a transient error rather than a silently dropped task. + +### Why this approach? + +The unconditional suspend is the bug; the minimal correct fix is to make suspend +conditional on ownership. A per-activation token decided by Sui's transaction +digest is the simplest possible ownership primitive: it is **unique by +construction**, needs no clock, no monotonic counter, and no coordination between +instances, and it eliminates the entire class of "two instances picked the same +value" races outright — there is no tiebreak to get wrong. Ownership is "whoever +wrote the token currently on record," resolved deterministically by consensus +ordering of writes to the shared registry. Non-aborting Move semantics keep the +shutdown path within its time budget and turn the expected race into a logged +no-op plus an on-chain event (the observability that was missing during the +incident). + +The recheck is **load-bearing, not cosmetic.** Because `status` is a single +per-record field, a rolling restart can deliver the graceful SIGTERM to the +_owner_ while a sibling is still alive: the owner suspends legitimately, the whole +record goes `Suspended`, and the surviving non-owner would otherwise idle forever +against it. The recheck reclaims within one interval, bounding downtime to ~60s +instead of days. The "killed owner, no sibling" case is intentionally **not** +handled in code — the orchestrator spins up a replacement that activates, writes a +new token, and overwrites the dead owner. + +--- + +## Alternatives Considered + +### Alternative 1: Monotonic version from the checkpoint clock + +**Description:** Store a `u64` version on the record sourced from the +`CheckpointClock`'s latest checkpoint sequence number; activation claims with +`max(stored, version)`, suspension requires `stored == version` ("highest version +wins"). This was the prior shape of this decision. + +**Why not:** It is strictly more machinery for the same guarantee. It introduces a +dependency on the checkpoint clock (and plumbing the sequence number through the +clock's state), an unseeded-clock guard so we never claim version `0`, and a +monotonic `max()` rule — and it still has a non-zero collision window (two +instances reading the same checkpoint) that has to be argued away. The transaction +digest removes the dependency, the guard, and the collision case entirely: it is +unique by protocol, so equality alone suffices and there is nothing to order. + +### Alternative 2: Owner check on suspend only (no always-claim, no recheck) + +**Description:** Add the token and the conditional suspend, but keep the "already +`Active` → no-op" activation and add no recheck. + +**Why not:** Insufficient. If the instance that receives the graceful SIGTERM is +the legitimate owner while a sibling lives, the owner suspends correctly and the +sibling — never having claimed ownership — is stranded against a `Suspended` +record. This reproduces the same outage by a different path. The always-claim +activation and the recheck close that gap. + +### Alternative 3: Leader-generated random token passed as an argument + +**Description:** Instead of `ctx.digest()`, the leader generates a random 32-byte +token off-chain and passes it into `activate_and_claim(token)`, storing it +optimistically. + +**Why not:** Functionally equivalent and collision-free as well, and marginally +simpler to capture (the leader already holds the value, no receipt read). We prefer +the transaction digest because it is self-generating on-chain, guaranteed unique by +the protocol's no-replay rule rather than by trusting an RNG, and idiomatic in this +codebase. Worth keeping as a fallback if reading the landed digest from the receipt +proves awkward. + +### Alternative 4: Dynamic field on the `Leader` UID for the token + +**Description:** Store the token in a Sui dynamic field keyed by leader cap ID +instead of a struct field, to avoid changing the `Leader` struct layout (Sui +package upgrades reject layout changes). + +**Why not:** Moot here — this ships in a framework version that redeploys the +registry and re-registers leaders, so a plain typed field is free and strictly +simpler. The dynamic-field approach would be the right call only if we needed an +in-place upgrade with no migration. + +### Alternative 5: Aborting Move guards + +**Description:** Make `suspend_if_token` abort on token mismatch. + +**Why not:** The suspend path `bail!`s on TX execution failure inside the bounded +shutdown-timeout budget; an aborting mismatch — the _expected_ path during a +rolling restart — would burn that budget and spam errors on every normal restart. +Non-abort + event is the right shape. + +### Alternative 6: Do nothing (manual recovery) + +**Description:** Keep recovering stuck leaders with a manual `scale 0→1`. + +**Why not:** The failure recurs on every instance refresh with unpredictable +timing, has produced multi-day outages, and silently removes backup-leader +redundancy. It is not self-correcting. + +--- + +## Consequences + +### Positive Consequences + +- A departing instance can no longer suspend a leader that another live instance + owns; rolling restarts stop stranding the record in `Suspended`. +- Ownership collisions are impossible by construction — the token is a unique + transaction digest, so there is no version-tie case to reason about and no clock + to depend on. +- Stuck-`Suspended` states self-heal within ~60s instead of requiring manual + intervention, covering both this race and unrelated causes. +- `LeaderSuspensionSkippedEvent` gives on-chain visibility into ownership + conflicts that was entirely absent during the incident. +- Ownership is derived from on-chain data with no inter-instance coordination, + shared lease store, or clock plumbing. + +### Negative Consequences + +- Changes span three layers (Move contract, nexus-sdk idents, leader runtime) and + require the SDK change to land before the leader compiles against it. +- The activation path must capture the landed activation transaction's digest from + its receipt and thread it onto `LeaderState`; a bug there leaves `claimed_token` + stale, which fails _safe_ (a non-matching token simply no-ops the suspend) but + could delay a legitimate suspend until the next recheck. +- A healthy single instance now issues a status TX on activation and one cheap read + per recheck interval; a suspended record drives a reclaim TX. + +### Neutral Consequences + +- The on-chain `Leader` struct layout changes — acceptable only because this ships + with a registry redeploy and leader re-registration. +- The token is an internal ownership value; `is_active` / `is_eligible` / ranking + remain keyed on `status` alone and are unaffected. +- Ownership is last-writer-wins, not "newest instance wins": a late-landing + activation from an older instance can take the token, but this only ever costs a + ≤60s recheck blip and never a stuck `Suspended`, because suspension stays + token-guarded and the recheck reclaims. +- A killed owner with no surviving sibling leaves the record `Active` with a dead + claimer until a replacement instance writes a new token — accepted, and resolved + by the orchestrator rather than in code. + +### Reversibility Assessment + +- **Reversibility:** Medium +- **Reversal cost:** Reverting requires backing out the Move field/functions (a + contract change), the SDK idents, and the runtime wiring. The runtime can fall + back to the unconditional `set_status` path mechanically, but undoing the + on-chain field needs a redeploy. +- **Point of no return:** The registry redeploy that introduces the field. After + that, removing the field is another layout change. + +--- + +## Context Evolution Tracking + +### Review Schedule + +- **Next review:** After observing leader status stability across several Cloud Run + rolling restarts and instance refreshes on testnet. +- **Review criteria:** Any recurrence of stuck-`Suspended`; frequency of + `LeaderSuspensionSkippedEvent` and recheck-driven reclaims in practice; whether + the 60s recheck interval is appropriately tuned; whether capturing the activation + digest from the receipt proved robust or warranted switching to a + leader-generated token (Alternative 3); whether the killed-owner-no-sibling + window ever bites in a single-instance configuration. + +--- diff --git a/nexus/adr/multi-coin-gas-payment.md b/nexus/adr/multi-coin-gas-payment.md new file mode 100644 index 0000000..6ff81df --- /dev/null +++ b/nexus/adr/multi-coin-gas-payment.md @@ -0,0 +1,210 @@ +# ADR-2: Multi-Coin Gas Payment for the Leader Gas Pool + +**Status:** Proposed +**Date:** 2026-06-03 +**Authors:** Pavel ([@kouks](https://github.com/kouks)) +**Deciders:** Pavel ([@kouks](https://github.com/kouks)) +**Consulted:** Engineering Team +**Informed:** Engineering Team + +**Constraint Tags:** Technical | Architectural + +--- + +## Context + +### What problem are we solving? + +The leader pays for Sui transactions from a pool of gas coins it owns. To avoid +equivocation on Sui, only one transaction may use a given gas coin at a time, so +the number of available coins directly bounds our transaction parallelism. Coins +are tracked in Redis (`be/leader/src/store/redis/redis_gas_manager.rs`); today +only the object ID is stored and the balance/version/digest are re-fetched from +Sui on every acquisition. + +During execution the leader claims payments from users into separate coins +(`be/leader/src/executor/handlers/walk_execution.rs`). A consolidator loop +(`be/leader/src/merchant/consolidator.rs`) merges these small claimed coins into +coins of at least a configured `gas_coin_min_balance_mist` threshold so they can +be reused to pay for transactions. + +With the current logic, the wallet drifts toward many coins each holding roughly +`gas_coin_min_balance_mist`. The consequence: **from time to time a transaction +needs more gas than the balance held on any single coin, and that transaction +fails.** Simply raising the threshold would let those transactions succeed but +would collapse parallelism by producing fewer, larger coins. + +### Relevant Constraints + +**Technical:** Sui forbids equivocation — a coin in flight in one transaction +must not be used by another. A single Sui transaction may, however, list **up to +256 gas coins**; the protocol "smashes" them, merging all into the first coin to +pay the fee. Transaction effects report `gas_used`, the gas object, and the +changed objects, which is enough to know what happened to the coins without a +follow-up query. + +**Architectural:** Coin selection must be driven by the transaction's actual gas +cost, which is only known after a dry run. The merchant already dry-runs before +acquiring a coin (see the dry-run/live submission split), so the budget is known +before coins are picked. + +### Architectural Position + +This decision affects the **leader gas pool** and the **merchant submission +pipeline**. Rather than sizing coins, we exploit Sui's multi-coin gas payment: +keep many low-balance coins and hand the chain as many as needed to cover a +transaction's budget. + +--- + +## Decision + +### What are we doing? + +We fund each transaction with **multiple gas coins** and make Redis the source of +truth for coin metadata, updated from transaction effects. + +1. **Multi-coin gas payment.** After the dry run computes the gas budget, the + merchant acquires as many pool coins as needed for their combined balance to + cover the budget, and submits the transaction listing all of them as the gas + payment. Sui smashes them into the first coin. + +1. **Per-transaction budget cap.** A configurable maximum gas budget (default + **10 SUI**) bounds a single transaction. It is also the dry-run placeholder + budget. If the budget computed from the dry run exceeds the cap, the + transaction fails right after the dry run (fatal). This keeps the number of + coins any one transaction needs well under the 256 limit, so the pool itself + may hold arbitrarily many coins. + +1. **Redis stores coin metadata.** The gas pool stores `{version, digest, + balance}` per coin, seeded at startup. After a transaction, effects tell us + the surviving (first) gas coin's new version/digest and its balance + (`sum(acquired balances) − effective gas used`); the smashed coins are + deleted. Redis is updated from this — **the per-acquisition on-chain refresh + is removed**. + +1. **Reconciliation.** Startup seeds the pool; transaction effects keep it + current on the hot path; the consolidator's existing discovery loop reconciles + drift and adds newly claimed coins (fetching their metadata once). + +1. **Ambiguous failures drop coins.** When a transaction is signed but its + execution outcome is unknown, the acquired coins are dropped from the pool; + the discovery loop re-adds them later with fresh on-chain metadata. + +1. **Dust floor retained.** `gas_coin_min_balance_mist` stays as the floor below + which coins are not tracked individually and are left for the consolidator to + merge. + +1. **Consolidator unchanged in spirit.** It still discovers claimed coins and + merges low-balance ones; it carries no coin-size or distribution logic. + +### Why this approach? + +Multi-coin gas payment lets a small transaction use one coin and a large one use +many, with no per-coin size ceiling and no rebalancing algorithm — the chain does +the merging at submission. Storing metadata in Redis and updating it from effects +removes a Sui round-trip from every acquisition and removes the coin-refresh code +path. The budget cap gives a single, easily-reasoned-about safety bound. The +result is markedly simpler than maintaining coin-size categories, especially in +the consolidator. + +--- + +## Alternatives Considered + +### Alternative 1: Coin size categories (S / M / L / XL) + +**Description:** Configure an ascending tuple of MIST thresholds mapped to size +categories. Partition the pool into per-size Redis sets, route each transaction +to the smallest size that can fund it (falling back to larger sizes), and have +the consolidator maintain a target distribution (more small coins, fewer large) +derived from total balance — merging and promoting coins to converge on it. + +**Why not:** It is substantially more complex for a strictly worse outcome. Size +selection is coarse (a transaction between thresholds over-provisions to the next +size up); the consolidator needs a bin-packing/promotion algorithm with +equivocation-safe claiming of in-flight coins; and the storage layer fragments +into per-size sets. Multi-coin gas payment achieves the same goal — funding +transactions of any cost while preserving parallelism — without any of this, +because the chain merges coins for us. This was the originally planned design and +was abandoned in favor of the decision above. + +### Alternative 2: Raise the single balance threshold + +**Description:** Keep one threshold but increase it so coins are always large +enough for expensive transactions. + +**Why not:** Fewer, larger coins directly reduce transaction parallelism, which +is bounded by the number of available coins. It trades one failure mode +(insufficient balance) for another (throughput collapse). + +### Alternative 3: Single balance-scored sorted set (ZSET) + +**Description:** Store all coins in one Redis sorted set scored by balance and +acquire the single smallest coin whose balance meets the budget via an atomic Lua +script. + +**Why not:** It still relies on a single coin covering the whole budget, so it +does not solve the large-transaction case any better than raising the threshold; +it only optimizes selection. + +### Alternative 4: Do nothing + +**Description:** Accept occasional failures of expensive transactions and rely on +retries. + +**Why not:** The failures are not self-correcting — the wallet's coin +distribution drifts steadily toward uniform threshold-sized coins, so large +transactions fail increasingly often with no recovery path. + +--- + +## Consequences + +### Positive Consequences + +- Expensive transactions succeed by combining as many coins as needed, while + parallelism is preserved through a population of small coins. +- An on-chain query is removed from every gas-coin acquisition; coin metadata + comes from transaction effects. +- The design is simple: no coin sizes, no distribution algorithm, no per-size + storage. The consolidator keeps its existing shape. +- A single config value (max gas budget) gives a clear safety bound and a + fast-fail for pathologically expensive transactions. + +### Negative Consequences + +- The gas pool and its Redis schema require a non-trivial refactor (storing and + updating coin metadata, acquiring sets of coins, settling from effects). +- Redis becomes the source of truth for balances/versions; a bug in + effects-based updates could let it drift until the discovery loop reconciles. +- Dropping coins on ambiguous failure has a small equivocation window if the + original transaction is still in flight when discovery re-adds the coin + (mitigated by a short quarantine before re-adding). + +### Neutral Consequences + +- A transaction may lock several coins at once, momentarily reducing parallelism + for large transactions — the intended trade-off. +- The pool may contain many coins; only the per-transaction count is bounded (by + the budget cap and the dust floor), not the pool size. + +### Reversibility Assessment + +- **Reversibility:** High +- **Reversal cost:** The change is internal to the leader; reverting to + single-coin acquisition with on-chain refresh is mechanical. +- **Point of no return:** None. + +--- + +## Context Evolution Tracking + +### Review Schedule + +- **Next review:** After observing transaction success rates and pool size in + production for a representative period. +- **Review criteria:** Transactions hitting the 256-coin limit despite the budget + cap; Redis balance drift in practice; need to revisit the budget cap default. + +--- diff --git a/nexus/guides/tool-communication.md b/nexus/guides/tool-communication.md index 8227e75..15703ff 100644 --- a/nexus/guides/tool-communication.md +++ b/nexus/guides/tool-communication.md @@ -325,7 +325,6 @@ Signature errors can also be caused by clock skew. Keep leader and tool clocks s Tool communication is primarily configured via these environment variables: - `ENVIRONMENT` -- `EXECUTOR_TOOL_INVOKE_TIMEOUT` - `EXECUTOR_TOOL_MAX_RESPONSE_BYTES` - `EXECUTOR_TOOL_TLS_ROOT_PEM_PATH` - `EXECUTOR_SIGNED_HTTP_MODE` diff --git a/nexus/packages/nexus-interface.md b/nexus/packages/nexus-interface.md index a6ed362..26d1cb7 100644 --- a/nexus/packages/nexus-interface.md +++ b/nexus/packages/nexus-interface.md @@ -1,53 +1,16 @@ # Nexus Interface -Nexus components are designed to be composed into "Talus Agent Packages" (TAPs) that are published on Sui. -TAPs are typically template Sui packages that are customized for a specific product. -They would include business logic relevant to the product and configurations for the Nexus components. - -The configurations typically take the form of `SuiObjectID`s which represent shared objects. -These shared objects, such as [`DAG`](workflow.md), are unforced to be invoked in situations described by the business logic. - -Some examples of what the business logic in a TAP could do: - -- check token supply to feature-gate a workflow -- require a payment component -- require a minimal time elapsed between invocations -- reward OP for reaching some step in the workflow - -## `V1` - -Nexus first iteration is focused on the workflow component. -To comply with the Nexus V1 Interface the TAP must: - -1. Register itself with the [Nexus leader](../crates/leader.md) by emitting `nexus_interface::v1::AnnounceInterfacePackageEvent` that - - has a generic parameter `W` which is a type located in the package and module that implements the interface. - - contains property `shared_objects: vector` is a list of `SuiObjectID`s that the leader will inject as args into each following public function call. - The list is ordered, that is the shared objects will be injected in the order they are listed. -2. Export `public fun worksheet(...${shared_objects}): ProofOfUID` - - Returns a stamp collector hot-potato. - - Collects execution confirmations by Nexus components such as `DAG`. - - Hot potato must be constructed with a type. - That can be achieved by calling `nexus_primitives::proof_of_uid::new_with_type`. - This type must be defined in the package and module that implements the interface. - Note that types don't change their type name when doing package upgrades. -3. Export `public fun confirm_tool_eval_for_walk(...${shared_objects}, ProofOfUID)` - - Consumes the hot-potato. - - Can verify that required confirmations have been collected. - - Invoked by the Nexus Leader. - -![Diagram showing Nexus V1 Interface flow](../images/nexus-interface-v1.png) - -### Events - -- `nexus_interface::v1::AnnounceInterfacePackageEvent` - - has the aforementioned generic parameter `W`; - - can be emitted many times by the same `W` in case the shared objects change, but delay should be tolerated between emitting this call and the BE syncing the changes; - - `shared_objects` is a list of `SuiObjectID`s that the leader will inject as args into each following public function call; - - when a package is upgraded, it must be called again with a newly published `W` - -### Upgrades - -Unfortunately, Sui does not give us a way to get current package ID. -When upgrading a package you must create a new type that serves as a witness. -This new type needs to be used when emitting `AnnounceInterfacePackageEvent` as its generic. -This event _also_ needs to be emitted with every package upgrade. +Nexus TAP execution now uses the standard TAP registry and endpoint model in +`nexus_interface::tap`. + +The standard TAP interface also creates an `AgentPaymentVault` for every new +agent. The vault holds agent-funded execution balances, tracks locked budget, +and is used by vault-backed `ExecutionPayment` settlement. Deposits are open; +withdrawals are authorized by the agent owner or operator through the TAP +registry. Payment finalization records source kind, source identity, locked +budget, consumed amount, and final state. + +The old v1 witness announcement flow is retired for active package authoring. +`nexus_interface::v1` remains only for non-witness shared-object reference and +configuration helpers that are still consumed by verifier compatibility and +historical decoding. diff --git a/nexus/packages/reference/nexus_interface/v1.md b/nexus/packages/reference/nexus_interface/v1.md index 3d3c206..918faea 100644 --- a/nexus/packages/reference/nexus_interface/v1.md +++ b/nexus/packages/reference/nexus_interface/v1.md @@ -1,87 +1,13 @@ + - +# `nexus_interface::v1` -# Module `(nexus_interface=0x0)::v1` +`v1` no longer exposes the retired witness announcement path for active package authoring. It remains as a compact compatibility module for shared-object reference and interface package configuration values used by verifier metadata and historical decoding. +Primary surfaces: +- `InterfacePackageConfig`: persisted shared-object reference configuration. +- `new_interface_package_config`: builds a config from shared object refs. +- ``: builds an immutable shared object reference. -- [Struct `AnnounceInterfacePackageEvent`](#(nexus_interface=0x0)_v1_AnnounceInterfacePackageEvent) -- [Function `announce_interface_package`](#(nexus_interface=0x0)_v1_announce_interface_package) - - -
use (nexus_primitives=0x0)::event;
-use std::ascii;
-use std::bcs;
-use std::option;
-use std::string;
-use std::vector;
-use sui::address;
-use sui::event;
-use sui::hex;
-use sui::object;
-use sui::tx_context;
-
- - - - - -## Struct `AnnounceInterfacePackageEvent` - - - -
public struct AnnounceInterfacePackageEvent<phantom W> has copy, drop
-
- - - -
-Fields - - -
-
-shared_objects: vector<sui::object::ID> -
-
-
-
- - -
- - - -## Function `announce_interface_package` - -Announce configuration for a package that implements V1 interface. - -The witness type W must be a type in the same package and module where the -interface is implemented. - -Whoever has access to W can announce a new list of shared_objects via -this function. - -If a package is upgraded, call this with a new witness type created on the -package upgrade. - - -
public fun announce_interface_package<W>(_witness: &W, shared_objects: vector<sui::object::ID>)
-
- - - -
-Implementation - - -
public fun announce_interface_package<W>(_witness: &W, shared_objects: vector<ID>) {
-    event::emit(AnnounceInterfacePackageEvent<W> {
-        shared_objects,
-    });
-}
-
- - - -
+New Talus Protocol execution should use [`nexus_interface::tap`](tap.md). Verifier packages should use [`nexus_interface::verifier_v1`](verifier_v1.md) for proof and result envelopes. diff --git a/nexus/packages/reference/nexus_primitives/proof_of_uid.md b/nexus/packages/reference/nexus_primitives/proof_of_uid.md index 98f2532..25269ee 100644 --- a/nexus/packages/reference/nexus_primitives/proof_of_uid.md +++ b/nexus/packages/reference/nexus_primitives/proof_of_uid.md @@ -204,7 +204,7 @@ Add a stamp to the proof.
public fun stamp(self: &mut ProofOfUID, uid: &UID) {
-    self.stamp_with_data(uid, vector::empty())
+    self.stamp_with_data(uid, vector[])
 }
 
diff --git a/nexus/packages/reference/nexus_registry/agent_registry.md b/nexus/packages/reference/nexus_registry/agent_registry.md new file mode 100644 index 0000000..85d3236 --- /dev/null +++ b/nexus/packages/reference/nexus_registry/agent_registry.md @@ -0,0 +1,19 @@ + + +# `nexus_registry::agent_registry` + +`agent_registry` stores Talus agent and skill records and provides the bridge between registered skills, workflow DAG execution, payment policy, agent payment vaults, and scheduler tasks. + +Primary surfaces: + +- `AgentRegistry`: shared registry object for agent records and default-agent executor state. +- `AgentRecord`: registry-side agent lifecycle and skill table state. +- `SkillRecord`: current skill contract, active flag, DAG binding, requirements, and revision metadata. +- `create_agent`, `attach_embedded_agent`: agent creation and registry binding. +- `create_skill`, `register_skill`, `register_skill_with_fixed_tools`: skill creation and compatibility registration helpers. +- `set_skill_active`, `set_agent_active`, `update_skill_description`, `update_dag`, `update_skill_policies`: lifecycle and revision management. +- `new_agent_skill_payment_for_execution`, `new_agent_skill_payment_from_vault_for_execution`, `new_default_dag_executor_payment_for_execution`: execution payment helpers. +- `schedule_skill_execution*`, `trigger_scheduled_skill_execution`, `create_scheduled_occurrence_payment`, `cancel_scheduled_skill_execution_from_agent_vault`: scheduler integration for address-funded and vault-funded skill execution. +- `default_dag_executor_*`: default agent executor inspection and payment helpers. + +See [Talus agent development](../../../TAP/agent-development.md), [Default agent](../../../TAP/default-agent.md), and [Agent skills and vaults](../../../TAP/agent-skills-and-vaults.md) for the conceptual workflow. diff --git a/nexus/packages/reference/nexus_workflow/dag.md b/nexus/packages/reference/nexus_workflow/dag.md index 342c039..a024524 100644 --- a/nexus/packages/reference/nexus_workflow/dag.md +++ b/nexus/packages/reference/nexus_workflow/dag.md @@ -2062,9 +2062,11 @@ Won't add the walk if it's already there. Advances the walk by evaluating the vertex and following the edges if possible. -
fun advance_walk(dag: &(nexus_workflow=0x0)::dag::DAG, execution: &mut (nexus_workflow=0x0)::dag::DAGExecution, request_walk_execution: &mut (nexus_workflow=0x0)::dag::RequestWalkExecution, walk_index: u64, expected_vertex: (nexus_workflow=0x0)::dag::Vertex, variant: (nexus_workflow=0x0)::dag::OutputVariant, variant_ports_to_data: sui::vec_map::VecMap<(nexus_workflow=0x0)::dag::OutputPort, (nexus_primitives=0x0)::data::NexusData>, ctx: &mut sui::tx_context::TxContext)
+
fun advance_walk(dag: &(nexus_workflow=0x0)::dag::DAG, execution: &mut (nexus_workflow=0x0)::dag::DAGExecution, request_walk_execution: &mut (nexus_workflow=0x0)::dag::RequestWalkExecution, walk_index: u64, expected_vertex: (nexus_workflow=0x0)::dag::RuntimeVertex, variant: (nexus_workflow=0x0)::dag::OutputVariant, variant_ports_to_data: sui::vec_map::VecMap<(nexus_workflow=0x0)::dag::OutputPort, (nexus_primitives=0x0)::data::NexusData>, failure_evidence_kind: (nexus_workflow=0x0)::dag::FailureEvidenceKind, clock: &sui::clock::Clock, ctx: &mut sui::tx_context::TxContext)
 
+If variant is _err_eval, tool evidence follows outgoing _err_eval edges first. When no such edge is available, the runtime resolves the effective post-failure action using vertex override, then DAG default, then terminate. Leader-evidence paths such as expired execution aborts are terminal-only and record TerminalErrEvalRecordedEvent instead of continuing through _err_eval edges. + @@ -2079,6 +2081,3 @@ Will emit an event if the execution is done.
fun if_finished_emit_final_event(self: &(nexus_workflow=0x0)::dag::DAGExecution)
 
- - - diff --git a/nexus/packages/reference/nexus_workflow/main.md b/nexus/packages/reference/nexus_workflow/main.md index 69c62af..b0559eb 100644 --- a/nexus/packages/reference/nexus_workflow/main.md +++ b/nexus/packages/reference/nexus_workflow/main.md @@ -13,7 +13,6 @@ use (nexus_primitives=0x0)::owner_cap; use (nexus_primitives=0x0)::proof_of_uid; use (nexus_workflow=0x0)::dag; -use (nexus_workflow=0x0)::default_tap; use (nexus_workflow=0x0)::gas; use (nexus_workflow=0x0)::leader_cap; use (nexus_workflow=0x0)::tool_registry; diff --git a/nexus/packages/reference/nexus_workflow/network_auth.md b/nexus/packages/reference/nexus_workflow/network_auth.md index b599169..9367b6e 100644 --- a/nexus/packages/reference/nexus_workflow/network_auth.md +++ b/nexus/packages/reference/nexus_workflow/network_auth.md @@ -33,37 +33,42 @@ Verifiers should accept signatures from the binding’s active key only (`KeyBin -- [Struct `NetworkAuth`](#(nexus_workflow=0x0)_network_auth_NetworkAuth) -- [Struct `KeyBinding`](#(nexus_workflow=0x0)_network_auth_KeyBinding) -- [Struct `KeyRecord`](#(nexus_workflow=0x0)_network_auth_KeyRecord) -- [Struct `ProofOfIdentity`](#(nexus_workflow=0x0)_network_auth_ProofOfIdentity) -- [Struct `ProofOfKey`](#(nexus_workflow=0x0)_network_auth_ProofOfKey) -- [Struct `NetworkAuthCreatedEvent`](#(nexus_workflow=0x0)_network_auth_NetworkAuthCreatedEvent) -- [Struct `KeyBindingCreatedEvent`](#(nexus_workflow=0x0)_network_auth_KeyBindingCreatedEvent) -- [Struct `KeyRegisteredEvent`](#(nexus_workflow=0x0)_network_auth_KeyRegisteredEvent) -- [Struct `KeyRevokedEvent`](#(nexus_workflow=0x0)_network_auth_KeyRevokedEvent) -- [Struct `ActiveKeyUpdatedEvent`](#(nexus_workflow=0x0)_network_auth_ActiveKeyUpdatedEvent) -- [Enum `IdentityKey`](#(nexus_workflow=0x0)_network_auth_IdentityKey) -- [Constants](#@Constants_0) -- [Function `new`](#(nexus_workflow=0x0)_network_auth_new) -- [Function `share`](#(nexus_workflow=0x0)_network_auth_share) -- [Function `prove_leader`](#(nexus_workflow=0x0)_network_auth_prove_leader) -- [Function `prove_offchain_tool`](#(nexus_workflow=0x0)_network_auth_prove_offchain_tool) -- [Function `proof_identity`](#(nexus_workflow=0x0)_network_auth_proof_identity) -- [Function `new_proof_of_key`](#(nexus_workflow=0x0)_network_auth_new_proof_of_key) -- [Function `binding_address`](#(nexus_workflow=0x0)_network_auth_binding_address) -- [Function `binding_exists`](#(nexus_workflow=0x0)_network_auth_binding_exists) -- [Function `create_binding`](#(nexus_workflow=0x0)_network_auth_create_binding) -- [Function `register_key`](#(nexus_workflow=0x0)_network_auth_register_key) -- [Function `revoke_key`](#(nexus_workflow=0x0)_network_auth_revoke_key) -- [Function `set_active_key`](#(nexus_workflow=0x0)_network_auth_set_active_key) -- [Function `key_binding_identity`](#(nexus_workflow=0x0)_network_auth_key_binding_identity) -- [Function `key_binding_active_key_id`](#(nexus_workflow=0x0)_network_auth_key_binding_active_key_id) -- [Function `key_binding_next_key_id`](#(nexus_workflow=0x0)_network_auth_key_binding_next_key_id) -- [Function `key_binding_key`](#(nexus_workflow=0x0)_network_auth_key_binding_key) -- [Function `assert_identity`](#(nexus_workflow=0x0)_network_auth_assert_identity) -- [Function `append_bytes`](#(nexus_workflow=0x0)_network_auth_append_bytes) -- [Function `clone_bytes`](#(nexus_workflow=0x0)_network_auth_clone_bytes) +- [Module `(nexus_workflow=0x0)::network_auth`](#module-nexus_workflow0x0network_auth) + - [Identity and key bindings](#identity-and-key-bindings) + - [Identities](#identities) + - [Proofs](#proofs) + - [Verification rule](#verification-rule) + - [Struct `NetworkAuth`](#struct-networkauth) + - [Struct `KeyBinding`](#struct-keybinding) + - [Struct `KeyRecord`](#struct-keyrecord) + - [Struct `ProofOfIdentity`](#struct-proofofidentity) + - [Struct `ProofOfKey`](#struct-proofofkey) + - [Struct `NetworkAuthCreatedEvent`](#struct-networkauthcreatedevent) + - [Struct `KeyBindingCreatedEvent`](#struct-keybindingcreatedevent) + - [Struct `KeyRegisteredEvent`](#struct-keyregisteredevent) + - [Struct `KeyRevokedEvent`](#struct-keyrevokedevent) + - [Struct `ActiveKeyUpdatedEvent`](#struct-activekeyupdatedevent) + - [Enum `IdentityKey`](#enum-identitykey) + - [Constants](#constants) + - [Function `new`](#function-new) + - [Function `share`](#function-share) + - [Function `prove_leader`](#function-prove_leader) + - [Function `prove_offchain_tool`](#function-prove_offchain_tool) + - [Function `proof_identity`](#function-proof_identity) + - [Function `new_proof_of_key`](#function-new_proof_of_key) + - [Function `binding_address`](#function-binding_address) + - [Function `binding_exists`](#function-binding_exists) + - [Function `create_binding`](#function-create_binding) + - [Function `register_key`](#function-register_key) + - [Function `revoke_key`](#function-revoke_key) + - [Function `set_active_key`](#function-set_active_key) + - [Function `key_binding_identity`](#function-key_binding_identity) + - [Function `key_binding_active_key_id`](#function-key_binding_active_key_id) + - [Function `key_binding_next_key_id`](#function-key_binding_next_key_id) + - [Function `key_binding_key`](#function-key_binding_key) + - [Function `assert_identity`](#function-assert_identity) + - [Function `append_bytes`](#function-append_bytes) + - [Function `clone_bytes`](#function-clone_bytes)
use (nexus_primitives=0x0)::event;
@@ -836,7 +841,7 @@ The owner cap is validated against the registry to bind the tool identity.
     owner_cap: &CloneableOwnerCap<OverTool>,
     fqn: AsciiString,
 ): ProofOfIdentity {
-    registry.assert_tool_owner(owner_cap, fqn);
+    registry.assert_tool_owner_generic(owner_cap, fqn);
     ProofOfIdentity {
         identity: IdentityKey::Tool {
             fqn,
diff --git a/nexus/packages/reference/nexus_workflow/tool_registry.md b/nexus/packages/reference/nexus_workflow/tool_registry.md
index 6d79369..e0f4726 100644
--- a/nexus/packages/reference/nexus_workflow/tool_registry.md
+++ b/nexus/packages/reference/nexus_workflow/tool_registry.md
@@ -14,42 +14,44 @@ It does **not** imply correctness, safety, availability, or that inputs/outputs
 are benign. Signed HTTP provides authenticity of responses; slashing is an
 after-the-fact enforcement mechanism.
 
-- [Struct `ToolRegistry`](#(nexus_workflow=0x0)_tool_registry_ToolRegistry)
-- [Struct `ToolStatus`](#(nexus_workflow=0x0)_tool_registry_ToolStatus)
-- [Struct `OffChainTool`](#(nexus_workflow=0x0)_tool_registry_OffChainTool)
-- [Struct `OverSlashing`](#(nexus_workflow=0x0)_tool_registry_OverSlashing)
-- [Struct `OverTool`](#(nexus_workflow=0x0)_tool_registry_OverTool)
-- [Struct `ToolRegistryCreatedEvent`](#(nexus_workflow=0x0)_tool_registry_ToolRegistryCreatedEvent)
-- [Struct `OffChainToolRegisteredEvent`](#(nexus_workflow=0x0)_tool_registry_OffChainToolRegisteredEvent)
-- [Struct `ToolUnregisteredEvent`](#(nexus_workflow=0x0)_tool_registry_ToolUnregisteredEvent)
-- [Struct `ToolSlashedEvent`](#(nexus_workflow=0x0)_tool_registry_ToolSlashedEvent)
-- [Struct `ToolStatusUpdatedEvent`](#(nexus_workflow=0x0)_tool_registry_ToolStatusUpdatedEvent)
-- [Constants](#@Constants_0)
-- [Function `new`](#(nexus_workflow=0x0)_tool_registry_new)
-- [Function `share`](#(nexus_workflow=0x0)_tool_registry_share)
-- [Function `slash_off_chain_tool`](#(nexus_workflow=0x0)_tool_registry_slash_off_chain_tool)
-- [Function `set_mist_collateral_to_lock`](#(nexus_workflow=0x0)_tool_registry_set_mist_collateral_to_lock)
-- [Function `set_lock_duration_ms`](#(nexus_workflow=0x0)_tool_registry_set_lock_duration_ms)
-- [Function `set_tool_status`](#(nexus_workflow=0x0)_tool_registry_set_tool_status)
-- [Function `register_off_chain_tool_for_self`](#(nexus_workflow=0x0)_tool_registry_register_off_chain_tool_for_self)
-  - [Slashing](#@Slashing_1)
-  - [Owner Cap](#@Owner_Cap_2)
-  - [Gas Tickets](#@Gas_Tickets_3)
-- [Function `register_off_chain_tool`](#(nexus_workflow=0x0)_tool_registry_register_off_chain_tool)
-- [Function `unregister_off_chain_tool`](#(nexus_workflow=0x0)_tool_registry_unregister_off_chain_tool)
-- [Function `claim_collateral_for_self`](#(nexus_workflow=0x0)_tool_registry_claim_collateral_for_self)
-- [Function `claim_collateral_for_off_chain_tool`](#(nexus_workflow=0x0)_tool_registry_claim_collateral_for_off_chain_tool)
-- [Function `deescalate`](#(nexus_workflow=0x0)_tool_registry_deescalate)
-- [Function `did_unregister_period_pass`](#(nexus_workflow=0x0)_tool_registry_did_unregister_period_pass)
-- [Function `assert_tool_registered`](#(nexus_workflow=0x0)_tool_registry_assert_tool_registered)
-- [Function `assert_tool_owner`](#(nexus_workflow=0x0)_tool_registry_assert_tool_owner)
-- [Function `assert_tool_owner_unchecked_generic`](#(nexus_workflow=0x0)_tool_registry_assert_tool_owner_unchecked_generic)
-- [Function `tool_status_unverified`](#(nexus_workflow=0x0)_tool_registry_tool_status_unverified)
-- [Function `tool_status_verified`](#(nexus_workflow=0x0)_tool_registry_tool_status_verified)
-- [Function `tool_status`](#(nexus_workflow=0x0)_tool_registry_tool_status)
-- [Function `tool_is_verified`](#(nexus_workflow=0x0)_tool_registry_tool_is_verified)
-- [Function `register_off_chain_tool_`](#(nexus_workflow=0x0)_tool_registry_register_off_chain_tool_)
-- [Function `did_unregister_period_pass_`](#(nexus_workflow=0x0)_tool_registry_did_unregister_period_pass_)
+- [Module `(nexus_workflow=0x0)::tool_registry`](#module-nexus_workflow0x0tool_registry)
+  - [Tool Verification Status](#tool-verification-status)
+  - [Struct `ToolRegistry`](#struct-toolregistry)
+  - [Enum `ToolStatus`](#enum-toolstatus)
+  - [Struct `OffChainTool`](#struct-offchaintool)
+  - [Struct `OverSlashing`](#struct-overslashing)
+  - [Struct `OverTool`](#struct-overtool)
+  - [Struct `ToolRegistryCreatedEvent`](#struct-toolregistrycreatedevent)
+  - [Struct `OffChainToolRegisteredEvent`](#struct-offchaintoolregisteredevent)
+  - [Struct `ToolUnregisteredEvent`](#struct-toolunregisteredevent)
+  - [Struct `ToolSlashedEvent`](#struct-toolslashedevent)
+  - [Struct `ToolStatusUpdatedEvent`](#struct-toolstatusupdatedevent)
+  - [Constants](#constants)
+  - [Function `new`](#function-new)
+  - [Function `share`](#function-share)
+  - [Function `slash_off_chain_tool`](#function-slash_off_chain_tool)
+  - [Function `set_mist_collateral_to_lock`](#function-set_mist_collateral_to_lock)
+  - [Function `set_lock_duration_ms`](#function-set_lock_duration_ms)
+  - [Function `set_tool_status`](#function-set_tool_status)
+  - [Function `register_off_chain_tool_for_self`](#function-register_off_chain_tool_for_self)
+    - [Slashing](#slashing)
+    - [Owner Cap](#owner-cap)
+    - [Gas Tickets](#gas-tickets)
+  - [Function `register_off_chain_tool`](#function-register_off_chain_tool)
+  - [Function `unregister_off_chain_tool`](#function-unregister_off_chain_tool)
+  - [Function `claim_collateral_for_self`](#function-claim_collateral_for_self)
+  - [Function `claim_collateral_for_off_chain_tool`](#function-claim_collateral_for_off_chain_tool)
+  - [Function `deescalate`](#function-deescalate)
+  - [Function `did_unregister_period_pass`](#function-did_unregister_period_pass)
+  - [Function `assert_tool_registered`](#function-assert_tool_registered)
+  - [Function `assert_tool_owner`](#function-assert_tool_owner)
+  - [Function `assert_tool_owner_generic`](#function-assert_tool_owner_generic)
+  - [Function `tool_status_unverified`](#function-tool_status_unverified)
+  - [Function `tool_status_verified`](#function-tool_status_verified)
+  - [Function `tool_status`](#function-tool_status)
+  - [Function `tool_is_verified`](#function-tool_is_verified)
+  - [Function `register_off_chain_tool_`](#function-register_off_chain_tool_)
+  - [Function `did_unregister_period_pass_`](#function-did_unregister_period_pass_)
 
 
use (nexus_primitives=0x0)::event;
 use (nexus_primitives=0x0)::owner_cap;
@@ -609,7 +611,7 @@ Return collateral to a tool owner.
 Returns a new [CloneableOwnerCap] for the given tool but with the given
 generic type that doesn't have any permissions within this module.
 
-See also [assert_tool_owner_unchecked_generic].
+See also [assert_tool_owner_generic].
 
 
public fun deescalate<T: drop>(self: &(nexus_workflow=0x0)::tool_registry::ToolRegistry, owner_cap: &(nexus_primitives=0x0)::owner_cap::CloneableOwnerCap<(nexus_workflow=0x0)::tool_registry::OverTool>, fqn: std::ascii::String, witness: T, ctx: &mut sui::tx_context::TxContext): (nexus_primitives=0x0)::owner_cap::CloneableOwnerCap<T>
 
@@ -635,16 +637,16 @@ See also [assert_tool_owner_unchecked_generic].
public fun assert_tool_owner(self: &(nexus_workflow=0x0)::tool_registry::ToolRegistry, owner_cap: &(nexus_primitives=0x0)::owner_cap::CloneableOwnerCap<(nexus_workflow=0x0)::tool_registry::OverTool>, fqn: std::ascii::String)
 
- + -## Function `assert_tool_owner_unchecked_generic` +## Function `assert_tool_owner_generic` Assert that the owner cap is for the given tool but allows any generic type to be used. See also [deescalate]. -
public fun assert_tool_owner_unchecked_generic<T>(self: &(nexus_workflow=0x0)::tool_registry::ToolRegistry, owner_cap: &(nexus_primitives=0x0)::owner_cap::CloneableOwnerCap<T>, fqn: std::ascii::String)
+
public fun assert_tool_owner_generic<T>(self: &(nexus_workflow=0x0)::tool_registry::ToolRegistry, owner_cap: &(nexus_primitives=0x0)::owner_cap::CloneableOwnerCap<T>, fqn: std::ascii::String)
 
diff --git a/nexus/packages/workflow.md b/nexus/packages/workflow.md index 07569c8..dc42848 100644 --- a/nexus/packages/workflow.md +++ b/nexus/packages/workflow.md @@ -174,6 +174,44 @@ Currently, the workflow engine reserves the following output variants: 1. Tool invocation failed 1. Output data could not be parsed or does not pass the tool's validation schema +### Post-failure action + +The workflow stores an optional `post_failure_action` on the DAG and an optional +`post_failure_action` on each executable vertex. + +Resolution order is: + +1. The executable vertex override. +1. The DAG default. +1. `Terminate`. + +Supported actions are: + +- `continue` +- `terminate` + +Fixture and DAG authoring use the same JSON shape. For example: + +```json +{ + "post_failure_action": "continue", + "vertices": [ + { + "name": "failable", + "post_failure_action": "terminate" + } + ] +} +``` + +Runtime behavior follows the simplified walk model: + +- Tool `_err_eval` is edge-first. If `_err_eval` edges exist, the engine follows them first. +- If no `_err_eval` edge exists, runtime resolves `continue` versus `terminate` using the precedence above. +- Timeout handling is terminal-only and records terminal `_err_eval` state regardless of configured `continue`. +- `terminate` records terminal `_err_eval`, aborts the walk, and cancels peer unfinished walks. +- When execution finishes with no active work remaining, consumed blocked walks normalize to `Cancelled`. + ### Entry ports Entry ports are input ports that must receive data from the client before execution can begin. These ports are explicitly defined via `with_entry_port` or `with_entry_port_in_group`. They cannot have any default values or incoming edges. diff --git a/nexus/tokenomics/gas-service.md b/nexus/tokenomics/gas-service.md index d8f4a06..86d3050 100644 --- a/nexus/tokenomics/gas-service.md +++ b/nexus/tokenomics/gas-service.md @@ -189,15 +189,26 @@ Used for custom gas strategies. Gas tickets and budgets can be associated with: 1. **Execution Scope** +1. **Agent Scope** 1. **Worksheet Type Scope** 1. **Invoker Address Scope** -When locking gas, the system searches in this order (specific -> general): +The historical generic workflow path searches in this order: ```move Execution → WorksheetType → InvokerAddress ``` +The standard TAP path searches in this order: + +```move +Execution → Agent → InvokerAddress +``` + +Agent scope is keyed by the standard TAP `Agent`. It lets a TAP package or +operator fund repeated execution for an agent without relying on worksheet-type +lookup or the current invoker's budget. + --- # Gas Locking Model diff --git a/nexus/tokenomics/tokenomics.md b/nexus/tokenomics/tokenomics.md index 6b64f30..6c6dd7f 100644 --- a/nexus/tokenomics/tokenomics.md +++ b/nexus/tokenomics/tokenomics.md @@ -49,7 +49,7 @@ The leader requires gas to submit tool invocation outputs onchain and will use g ### Library of gas extensions -The Talus Labs team will provide some gas exentsion Move packages that can will be able to make up the bulk of use cases, including gas tickets with expiry and limited number of invocations. +The Talus Labs team will provide some gas extension Move packages that can will be able to make up the bulk of use cases, including gas tickets with expiry and limited number of invocations. However, the tool developers are free to write custom gas extensions Move packages that correspond to custom mode of operation for their gas tickets.