Proposal: 修复上游 #326 Wasm `doAfterVmCallActions` 跨阶段回调 drain — stage-correct drain in `higress-group/proxy-wasm-cpp-host` || Proposal: Fix upstream #326 Wasm `doAfterVmCallActions` cross-stage callback drain — stage-correct drain in `higress-group/proxy-wasm-cpp-host`

[johnlanni]

Proposal: 修复上游 #326 Wasm doAfterVmCallActions 跨阶段回调 drain — stage-correct drain in higress-group/proxy-wasm-cpp-host

Source bug: proxy-wasm/proxy-wasm-cpp-host#326 — doAfterVmCallActions 的共享延迟队列在 sendLocalReply → onResponseHeaders 重入时，会把请求阶段排入的 continueDecoding 在响应阶段 drain 掉（阶段错位执行）。
本 proposal 是 #4064 上 SPEC-003 / QUESTION-002 指向的正式后续。与 #4034（CPU 死循环）分离：#4064 的 Layer A（drain-to-local）不修复 #326，需单独 change。
本次迭代只做立项 / scoping，不实现修复。 无 design/implement/TASK 工件。SPEC/QUESTION 以 typed comment 挂在本 issue 下。

关联项目

项目	GitHub	关键文件
Higress proxy-wasm-cpp-host fork	https://github.com/higress-group/proxy-wasm-cpp-host	include/proxy-wasm/wasm.h、include/proxy-wasm/context.h、src/context.cc、src/wasm.cc
Higress envoy fork	https://github.com/higress-group/envoy	source/extensions/common/wasm/context.cc
上游 proxy-wasm-cpp-host	https://github.com/proxy-wasm/proxy-wasm-cpp-host	issue #326、（已关闭未合并）PR #423

引用版本：pwh fork pinned commit 8549cf6374835d225edf67e584cb8d8d8a0fc256。

关联工件

• Proposal #4064、Design #4070 — Wasm 插件死循环：onRedisCallFailure + injectEncodedDataToFilterChain 嵌套导致 Envoy CPU 100% 跑满 #4034 CPU 死循环的 Layer A（drain-to-local）。
• SPEC-003（Update the leveled-configuration model in the README files of Wasm plugins #326 阶段错位需求）、QUESTION-002（Update the leveled-configuration model in the README files of Wasm plugins #326 是否入 Wasm 插件死循环：onRedisCallFailure + injectEncodedDataToFilterChain 嵌套导致 Envoy CPU 100% 跑满 #4034 迭代；默认 X1 = 单独跟踪）。本 proposal 即 QUESTION-002 默认解所指向的独立 change，并在此延续 / supersede SPEC-003。

结论

#326 是一个真实且与 #4034 不同的 bug——症状是跨阶段回调 drain（stage-mismatch），不是 CPU 死循环。已通过 fork pinned 代码追踪确认：

• Layer A（Proposal: 修复 #4034 Wasm 插件 CPU 死循环 — drain-to-local in higress-group/proxy-wasm-cpp-host || Proposal: Fix #4034 Wasm plug-in CPU infinite loop — drain-to-local in higress-group/proxy-wasm-cpp-host #4064/Design: drain-to-local in WasmBase::doAfterVmCallActions (Layer A, #4034) #4070 的 drain-to-local）不修复 Update the leveled-configuration model in the README files of Wasm plugins #326——它让 drain 循环重入安全（终结 CPU spin），但保留了 FIFO drain 语义，请求阶段排入的动作照样会在响应阶段的 drain 里被执行。
• 根因是队列作用域 + 无阶段过滤：after_vm_call_actions_ 是 WasmBase 上的每-VM 共享 std::deque，~DeferAfterCallActions 无条件 drain，且同一个 Context 实例同时服务 decode 与 encode 阶段——所以把队列改成 per-context 也不能修复（逃逸动作与 drain 共用同一 context）。

因此 #326 需要自己的 change。本次迭代只立项、只 scoping、不实现，与上游 #326 thread 协同设计，待 Layer A（#4064/#4070）落地后再实施。

Why（根因追踪，file:line）

• 延迟动作队列 after_vm_call_actions_ 是 WasmBase 的成员（每-VM 共享，非 per-context）——proxy-wasm-cpp-host include/proxy-wasm/wasm.h:338；enqueue/drain 逻辑在 wasm.h:158-169。
• DeferAfterCallActions(ContextBase*) 构造时丢弃 context、只保留 WasmBase*（include/proxy-wasm/context.h:455）；~DeferAfterCallActions 无条件调用 wasm_->doAfterVmCallActions()（src/context.cc:51-54）。
• 请求 / decoder 阶段会把 sendLocalReply（higress-group/envoy source/extensions/common/wasm/context.cc:2058）和 continueDecoding（context.cc:1929）排入同一个共享 deque。
• drain 先跑 sendLocalReply，它同步驱动 encodeHeaders → onResponseHeaders（context.cc:2254）→ 触发嵌套的 ~DeferAfterCallActions → drain 同一个 deque，于是仍排在队列里的请求阶段 continueDecoding 在响应阶段的 VM call 内被执行 → 阶段错位。
• 同一个 Context 实例既服务 decode（context.cc:2150）又服务 encode（context.cc:2254）——这正是 per-context 队列无法修复的原因：逃逸动作与 drain 共用同一 context。

Fix candidates（候选修法 + 排序 + blast-radius）

1. Stage-tagging（最小可辩护，仍属 MEDIUM 风险）— 领先候选

引入 stage 概念并按当前 drain stage 过滤。没有现成的 phase enum 可复用：ContextBase/envoy Context 上并未持久化阶段——WasmStreamType{Request,Response,Downstream,Upstream} 只作为 continueStream 的参数存在、从不落盘；唯一的线程局部状态是 wasm_vm.h:382-387 的 current_context_/effective_context_id_，那是 context 身份、不是 phase。

因此最小可辩护补丁：

• 新增 WasmBase::current_stage_，在 DeferAfterCallActions 里 save/restore（借助 auto-tag 保持约 20 处 addAfterVmCallAction 调用签名不变）；
• 把 after_vm_call_actions_ 改成 deque<pair<stage, fn>>；
• doAfterVmCallActions 只执行 tag 与当前 drain stage 匹配的动作，始终执行 stage-agnostic 的强制清理，teardown 时 flush 全部；

成本： ~3 个 pwh 文件 + envoy fork 在约 40 处 DeferAfterCallActions actions(this); 构造点做注解 + 分类约 20 处 enqueue 调用点 + ~80 行逻辑。

2. Per-context 队列（把 deque 移到 ContextBase）— 拒绝

不修复 #326：逃逸动作与 drain 共用同一个 Context。Reject。

3. Terminal SendHttpResponse + 抑制 self-reply 回调（PiotrSikora 方向）— 最正确但破坏性最大

让 SendHttpResponse 成为终止操作（其后不再跑 wasm），并且永不为插件自身的 local reply 触发 onResponse*。最贴近根因，但破坏性且在 Envoy 侧，改动面最大。

Blast-radius 诚实评估（stage-tagging）

该改动只是部分可控：

• 机械部分 OK： 无 wasm ABI 变更 → 插件不受影响；约 40 处 Defer 构造点的改动是编译器可捕获的；与 Layer A 正交。
• 风险集中在：
  • (a) 分类风险： 约 20 处 enqueue 调用点必须人工区分为「阶段敏感的续跑」vs「阶段无关的强制清理」（context.cc:1260/1287/2367/2398/2423/2465 的 handler-map erase、pwh wasm.cc:532 的 shutdown-handle delete）。清理误分类 → 资源泄漏；续跑误分类 → 请求挂起；两者都是静默且时序相关，难以单测覆盖。
  • (b) 双仓协同： 同时触及 pwh fork 与 envoy fork，必须同改、同 pin、同发（协同成本高于 Layer A 的单文件改动）。
  • (c) teardown-flush 是新行为，需要专门的边界用例测试。
  • (d) 与上游意愿分歧： 需长期携带 fork patch。
• 与 Layer A 的交互： 正交、可安全叠加。唯一的协同约束是 stage 过滤器必须比对当前 drain stage（在嵌套间 restore），不是 enqueue 时刻快照的 stage。

Upstream 先例（诚实记录）

上游 PR proxy-wasm/proxy-wasm-cpp-host#423 已关闭未合并。它没有实现 stage-tagging，只改了 exports.cc 里 send_local_response 的错误传播（envoyproxy/envoy#28826 的一个较窄症状），并破坏了 host-ABI 契约（Rust SDK 在 error 时 panic）。维护者 PiotrSikora 主张正确修法是 (a) 让 SendHttpResponse 终止化（其后不跑 wasm）、(b) 永不为插件自身 local reply 触发 onResponse*——两者都破坏性且在 Envoy 侧。维护者 johnlanni 提议 stage-tagging。未达成共识。

Decision（本次迭代）

• 不实现。 仅立项 / scoping。
• 与上游 Update the leveled-configuration model in the README files of Wasm plugins #326 thread 协同设计。
• 在 Layer A（Proposal: 修复 #4034 Wasm 插件 CPU 死循环 — drain-to-local in higress-group/proxy-wasm-cpp-host || Proposal: Fix #4034 Wasm plug-in CPU infinite loop — drain-to-local in higress-group/proxy-wasm-cpp-host #4064/Design: drain-to-local in WasmBase::doAfterVmCallActions (Layer A, #4034) #4070）落地之后再实施。
• 推荐 stage-tagging「最小可辩护补丁」为领先候选，但标注为 medium-risk；并记录上游 terminal-SendHttpResponse 替代方案。

Scope

In（本次仅做设计 / 决策，不写码）：

• higress-group/proxy-wasm-cpp-host 中 stage-correct drain 的设计与决策（+ envoy fork 的 enqueue 注解）。

Deferred：

• 实际实现（stage-tagging 补丁本身）——待本 change 的 design/implement 迭代。

Out：

• Envoy 侧 terminal-SendHttpResponse 重设计（上游、破坏性）。
• 重新讨论 Layer A（Proposal: 修复 #4034 Wasm 插件 CPU 死循环 — drain-to-local in higress-group/proxy-wasm-cpp-host || Proposal: Fix #4034 Wasm plug-in CPU infinite loop — drain-to-local in higress-group/proxy-wasm-cpp-host #4064/Design: drain-to-local in WasmBase::doAfterVmCallActions (Layer A, #4034) #4070 已定）。

Open questions

• QUESTION（本 issue） — 修法路线：stage-tagging（在 fork 中先原型）vs 上游 terminal-reply 重设计先行？非阻塞，默认「先在 fork 中原型 stage-tagging，落地前与上游 Update the leveled-configuration model in the README files of Wasm plugins #326 thread 协同设计」。
• 推 upstream 优先 vs 长期携带 fork patch？
• backport 目标（release 分支）待定。

---

Proposal: Fix upstream #326 Wasm doAfterVmCallActions cross-stage callback drain — stage-correct drain in higress-group/proxy-wasm-cpp-host

Source bug: proxy-wasm/proxy-wasm-cpp-host#326 — The shared delay queue of doAfterVmCallActions will queue the request phase into continueDecoding when sendLocalReply → onResponseHeaders re-enters. Drain it in the response phase (stage misalignment execution).
This proposal is the official follow-up to SPEC-003 / QUESTION-002 on #4064. Separated from #4034 (CPU infinite loop): Layer A (drain-to-local) of #4064 not fixed #326, needs to be changed separately.
**This iteration only does project establishment/scoping and does not implement repairs. ** No design/implement/TASK artifact. SPEC/QUESTION is hung under this issue as a typed comment.

Related projects

Project	GitHub	Key Files
Higress proxy-wasm-cpp-host fork	https://github.com/higress-group/proxy-wasm-cpp-host	include/proxy-wasm/wasm.h, include/proxy-wasm/context.h, src/context.cc, src/wasm.cc
Higress envoy fork	https://github.com/higress-group/envoy	source/extensions/common/wasm/context.cc
Upstream proxy-wasm-cpp-host	https://github.com/proxy-wasm/proxy-wasm-cpp-host	issue #326, (closed but not merged) PR #423

Reference version: pwh fork pinned commit 8549cf6374835d225edf67e584cb8d8d8a0fc256.

Associated artifacts

• Proposal #4064, Design #4070 — Wasm 插件死循环：onRedisCallFailure + injectEncodedDataToFilterChain 嵌套导致 Envoy CPU 100% 跑满 #4034 Layer A of CPU infinite loop (drain-to-local).
• SPEC-003(Update the leveled-configuration model in the README files of Wasm plugins #326 Stage misalignment requirements), QUESTION-002(Update the leveled-configuration model in the README files of Wasm plugins #326 Whether to enter Wasm 插件死循环：onRedisCallFailure + injectEncodedDataToFilterChain 嵌套导致 Envoy CPU 100% 跑满 #4034 iteration; default X1 = tracked alone). This proposal is an independent change pointed to by the default solution of QUESTION-002, and continues here / supersede SPEC-003.

Conclusion

#326 is a real bug and different from #4034** - the symptom is cross-stage callback drain (stage-mismatch), not a CPU infinite loop. Confirmed through fork pinned code tracking:

• Layer A (drain-to-local of Proposal: 修复 #4034 Wasm 插件 CPU 死循环 — drain-to-local in higress-group/proxy-wasm-cpp-host || Proposal: Fix #4034 Wasm plug-in CPU infinite loop — drain-to-local in higress-group/proxy-wasm-cpp-host #4064/Design: drain-to-local in WasmBase::doAfterVmCallActions (Layer A, #4034) #4070) does not fix Update the leveled-configuration model in the README files of Wasm plugins #326 - it makes drain loop reentrancy safe (terminating CPU spin), but retains FIFO drain semantics, and actions queued in the request phase will still be executed in the drain in the response phase.
• The root cause is queue scope + stageless filtering: after_vm_call_actions_ is a per-VM shared std::deque on WasmBase, ~DeferAfterCallActions unconditionally drains, and the same Context instance serves both decode and encode phases** - so changing the queue to per-context cannot fix it (escape actions and drain share the same context).

Therefore #326 requires its own change. This iteration is only project establishment, only scoping, not implementation. It is designed in collaboration with the upstream #326 thread and will be implemented after Layer A (#4064/#4070) is implemented.

Why (root cause tracking, file: line)

• Deferred action queue after_vm_call_actions_ is a member of WasmBase (per-VM shared, not per-context) - proxy-wasm-cpp-host include/proxy-wasm/wasm.h:338; enqueue/drain logic is in wasm.h:158-169.
• DeferAfterCallActions(ContextBase*) discards context during construction and only retains WasmBase*** (include/proxy-wasm/context.h:455); ~DeferAfterCallActions unconditionally calls wasm_->doAfterVmCallActions() (src/context.cc:51-54).
• The request/decoder phase will enqueue sendLocalReply (higress-group/envoy source/extensions/common/wasm/context.cc:2058)** and continueDecoding (context.cc:1929`) into the same shared deque.
• drain runs sendLocalReply first, which synchronously drives encodeHeaders → onResponseHeaders (context.cc:2254) → triggers nested ~DeferAfterCallActions → drain the same deque, so the request phase continueDecoding that is still queued is executed within the VM call in the **response phase → phase misalignment.
• The same Context instance serves both decode (context.cc:2150) and encode (context.cc:2254) - this is why the per-context queue cannot be repaired: the escape action shares the same context as drain.

Fix candidates (candidate fix + sort + blast-radius)

1. Stage-tagging (minimally defensible, still a MEDIUM risk) – leading candidate

Introduce the stage concept and filter by current drain stage. There is no ready-made phase enum that can be reused: ContextBase/envoy Context does not have a persistence phase - WasmStreamType{Request,Response,Downstream,Upstream} only exists as a parameter of continueStream and is never saved; the only thread local state is wasm_vm.h:382-387 current_context_/effective_context_id_, that is context identity, not phase.

So the minimal defensible patch:

• Added WasmBase::current_stage_, save/restore in DeferAfterCallActions (use auto-tag to keep about 20 addAfterVmCallAction call signatures unchanged);
• Change after_vm_call_actions_ to deque<pair<stage, fn>>;
• doAfterVmCallActions only executes actions whose tag matches the current drain stage, always executes stage-agnostic forced cleanup, and flushes all when teardown occurs;

Cost: ~3 pwh files + envoy fork at about 40 DeferAfterCallActions actions(this); construction points annotated + classification at about 20 enqueue call points + ~80 lines of logic.

2. Per-context queue (move deque to ContextBase) - Rejected

Does not fix #326: Escape action and drain share the same Context. Reject.

3. Terminal SendHttpResponse + suppress self-reply callback (PiotrSikora direction) - the most correct but most destructive

Make SendHttpResponse the final action (no wasm is run afterward), and never trigger onResponse* for the plugin's own local reply. Closest to the root cause, but destructive and on the Envoy side with the greatest scope for change.

Blast-radius honest assessment (stage-tagging)

This change is only partially controllable:

• Mechanical OK: No wasm ABI changes → Plugins are not affected; about 40 Defer construction point changes are compiler catchable; orthogonal to Layer A.
• Risk concentration in:
  • **(a) Classification risk: ** About 20 enqueue call points must be manually distinguished into "stage-sensitive continuation" vs "stage-independent forced cleanup" (handler-map erase of context.cc:1260/1287/2367/2398/2423/2465, shutdown-handle delete of pwh wasm.cc:532). Cleanup misclassification → resource leakage; continuation misclassification → request suspension; both are silent and timing-related, and are difficult to cover with a single test.
  • **(b) Dual warehouse collaboration: ** If pwh fork and envoy fork are involved at the same time, they must be modified, pinned and released at the same time (the collaboration cost is higher than single file changes in Layer A).
  • (c) teardown-flush is new behavior and requires specialized edge case testing.
  • **(d) Divergence from upstream wishes: ** Need to carry fork patch for a long time.
• Interaction with Layer A: Orthogonal, safe to stack. The only co-constraint is that the stage filter must be compared to the current drain stage (restored between nests), not the stage at the enqueue time snapshot.

Upstream precedent (honest record)

Upstream PR proxy-wasm/proxy-wasm-cpp-host#423 Closed and not merged. It does not implement stage-tagging, only changes the error propagation of send_local_response in exports.cc (a narrow symptom of envoyproxy/envoy#28826), and breaks the host-ABI contract (Rust SDK panics on errors). Maintainer PiotrSikora advocates that the correct fix is ​​to (a) let SendHttpResponse terminate (not run wasm afterwards), and (b) never trigger onResponse* for the plugin's own local reply - both are destructive and on the Envoy side. Maintainer johnlanni proposed stage-tagging. **No consensus reached. **

Decision (this iteration)

• **Not implemented. **Project establishment/scoping only.
• Co-design with upstream Update the leveled-configuration model in the README files of Wasm plugins #326 thread.
• Implement it after Layer A (Proposal: 修复 #4034 Wasm 插件 CPU 死循环 — drain-to-local in higress-group/proxy-wasm-cpp-host || Proposal: Fix #4034 Wasm plug-in CPU infinite loop — drain-to-local in higress-group/proxy-wasm-cpp-host #4064/Design: drain-to-local in WasmBase::doAfterVmCallActions (Layer A, #4034) #4070) is implemented.
• Recommend stage-tagging "minimum defensible patch" as the leading candidate, but marked as medium-risk; and document the upstream terminal-SendHttpResponse alternative.

Scope

In (only design/decision-making this time, no coding):

• Design and decisions of stage-correct drain in higress-group/proxy-wasm-cpp-host (+ enqueue annotation of envoy fork).

Deferred：

• Actual implementation (stage-tagging patch itself) - pending design/implementation iteration of this change.

Out:

• Envoy side terminal-SendHttpResponse redesign (upstream, destructive).
• Revisit Layer A (Proposal: 修复 #4034 Wasm 插件 CPU 死循环 — drain-to-local in higress-group/proxy-wasm-cpp-host || Proposal: Fix #4034 Wasm plug-in CPU infinite loop — drain-to-local in higress-group/proxy-wasm-cpp-host #4064/Design: drain-to-local in WasmBase::doAfterVmCallActions (Layer A, #4034) #4070 finalized).

Open questions

• QUESTION (this issue) — Method revision route: stage-tagging (prototype in fork first) vs upstream terminal-reply redesign first? Non-blocking, the default is "prototype stage-tagging in fork first, and collaborate with the upstream Update the leveled-configuration model in the README files of Wasm plugins #326 thread before launching".
• Push upstream first vs carry fork patches long term?
• The backport target (release branch) is to be determined.

[johnlanni]

Agent: Coordinator
Type: SPEC
ID: SPEC-001
Status: confirmed
Scope: proxy-wasm-cpp-host:stage-correct-drain
Links:

• Proposal Issue: N/A
• Design Issue: N/A
• Implement Issue: N/A
• Related Comments: N/A
• PR: N/A

Requirement: doAfterVmCallActions MUST drain each deferred action in the stage in which it was enqueued (stage-correct drain)

WasmBase::doAfterVmCallActions SHALL run each deferred after_vm_call_actions_ entry in the same VM-callback stage/phase in which it was enqueued. A drain triggered from a response/encoder-phase VM call MUST NOT execute a stage-sensitive continuation (e.g. decoder_callbacks_->continueDecoding()) that was enqueued during the request/decoder phase. Stage-agnostic mandatory cleanups MUST always run regardless of the current drain stage.

This requirement continues and supersedes SPEC-003 on #4064: SPEC-003 captured the #326 stage-mismatch as a deferred item under the #4034 change; this SPEC is the authoritative requirement for the dedicated wasm-aftervmcall-stage-mismatch-326 change. The current queue is a single per-VM FIFO on WasmBase (include/proxy-wasm/wasm.h:338) drained unconditionally by ~DeferAfterCallActions (src/context.cc:51-54), so the current implementation violates this requirement.

Scenario: request-phase continueDecoding MUST NOT fire during response-phase sendLocalReply → onResponseHeaders reentry

• WHEN a decoder-side callback enqueues decoder_callbacks_->continueDecoding() via addAfterVmCallAction (envoy context.cc:1929) AND, before the outer ~DeferAfterCallActions drains, sendLocalReply is also enqueued (envoy context.cc:2058) onto the same shared after_vm_call_actions_ deque.
• THEN when the drain runs sendLocalReply, which synchronously drives encodeHeaders → onResponseHeaders (envoy context.cc:2254) and thereby triggers a nested ~DeferAfterCallActions drain of the SAME deque, the still-queued request-phase continueDecoding action SHALL NOT be executed inside that response-phase VM call.
• SO THAT decoder_callbacks_->continueDecoding() is never invoked outside the decoder phase — avoiding no-op resumes, double-resumes, internal-assertion failures, or use-after-free of decoder filter-manager state.

Scenario: stage-agnostic mandatory cleanups MUST still always run

• WHEN a deferred action is a stage-agnostic mandatory cleanup rather than a stage-sensitive continuation — e.g. a handler-map erase (envoy context.cc:1260/1287/2367/2398/2423/2465) or a shutdown-handle delete (pwh wasm.cc:532).
• THEN doAfterVmCallActions SHALL execute that action on the next drain regardless of which stage the drain is running in, and teardown SHALL flush any remaining such actions.
• SO THAT the stage filter never strands a mandatory cleanup (which would leak resources), i.e. the stage-correct filter only gates stage-sensitive continuations, not cleanups.

Scenario: stage filter MUST compare against the current drain stage, not the enqueue-time snapshot

• WHEN drains are nested (an outer decoder-phase drain synchronously enters a response-phase VM call whose destructor starts an inner drain).
• THEN the stage used to admit/skip an action SHALL be the stage of the currently-executing drain frame (save/restore across nesting), NOT a stage snapshot captured at enqueue time only used for equality.
• SO THAT Layer A (drain-to-local, Proposal: 修复 #4034 Wasm 插件 CPU 死循环 — drain-to-local in higress-group/proxy-wasm-cpp-host || Proposal: Fix #4034 Wasm plug-in CPU infinite loop — drain-to-local in higress-group/proxy-wasm-cpp-host #4064/Design: drain-to-local in WasmBase::doAfterVmCallActions (Layer A, #4034) #4070) and this stage-correct drain compose safely: Layer A guarantees reentry-safety; this SPEC guarantees stage-correctness; the two are orthogonal and must both hold.

Non-goals / relationship to Layer A

• Layer A (SPEC-001 / drain-to-local, Proposal: 修复 #4034 Wasm 插件 CPU 死循环 — drain-to-local in higress-group/proxy-wasm-cpp-host || Proposal: Fix #4034 Wasm plug-in CPU infinite loop — drain-to-local in higress-group/proxy-wasm-cpp-host #4064/Design: drain-to-local in WasmBase::doAfterVmCallActions (Layer A, #4034) #4070) terminates the Wasm 插件死循环：onRedisCallFailure + injectEncodedDataToFilterChain 嵌套导致 Envoy CPU 100% 跑满 #4034 CPU-spin but does NOT satisfy this requirement (FIFO drain preserves cross-stage execution). This SPEC MUST NOT regress Layer A's reentry-safe drain.
• The Envoy-side terminal-SendHttpResponse redesign (upstream, breaking) is out of scope for this SPEC; it is an alternative implementation strategy, not a change to this requirement.

References

• Upstream: proxy-wasm/proxy-wasm-cpp-host#326; closed-unmerged PR #423.
• pwh fork (pinned 8549cf6374835d225edf67e584cb8d8d8a0fc256): include/proxy-wasm/wasm.h:158-169,338, include/proxy-wasm/context.h:455, src/context.cc:51-54, src/wasm.cc:532.
• envoy fork: source/extensions/common/wasm/context.cc:1929,2058,2150,2254.
• Continues: SPEC-003 on #4064; scope decision QUESTION-002 on #4064.

[johnlanni]

Agent: Coordinator
Type: QUESTION
ID: QUESTION-001
Status: draft
Scope: 326-fix-approach
Links:

• Proposal Issue: Proposal: 修复上游 #326 Wasm doAfterVmCallActions 跨阶段回调 drain — stage-correct drain in higress-group/proxy-wasm-cpp-host || Proposal: Fix upstream #326 Wasm doAfterVmCallActions cross-stage callback drain — stage-correct drain in higress-group/proxy-wasm-cpp-host #4073
• Design Issue: N/A
• Implement Issue: N/A
• Related Comments: Proposal: 修复上游 #326 Wasm doAfterVmCallActions 跨阶段回调 drain — stage-correct drain in higress-group/proxy-wasm-cpp-host || Proposal: Fix upstream #326 Wasm doAfterVmCallActions cross-stage callback drain — stage-correct drain in higress-group/proxy-wasm-cpp-host #4073 (comment)
• PR: N/A

Question

Question

Which fix approach should the wasm-aftervmcall-stage-mismatch-326 change pursue for the stage-correct drain (SPEC-001)?

Two viable directions (per-context queue is rejected — same Context serves both decode and encode, so it does not fix #326):

• Option A (default): Prototype stage-tagging in the Higress forks now (higress-group/proxy-wasm-cpp-host + higress-group/envoy), co-design with the upstream #326 thread before landing. Smallest defensible patch; no wasm ABI change; orthogonal to Layer A. Medium-risk (manual classification of ~20 enqueue callsites; dual-repo co-pin/co-release; new teardown-flush behavior).
• Option B: Push the upstream terminal-SendHttpResponse + suppress self-reply callbacks redesign (PiotrSikora's direction) first, and only carry a fork patch if upstream stalls. Most correct but breaking and Envoy-side; largest blast radius; historically no consensus (see closed-unmerged PR #423).

Sub-decisions gated by this: push upstream-first vs. carry a long-lived fork patch; backport targets (release branches).

Non-blocking. This does not block filing/scoping this iteration (no implementation this iteration). It gates the eventual design/implement approach.

Default assumption if unresolved: Option A — prototype stage-tagging in the fork, co-design with the upstream #326 thread before landing, and land only after Layer A (#4064/#4070) ships.

Blocking

false

Default Assumption

Option A — prototype stage-tagging in the fork, co-design with upstream #326 thread before landing; land after Layer A (#4064/#4070).

Resolution Log

• Pending.

[johnlanni]

layer A修复已经足够了

---

layer A repair is enough