Skip to content

fix(judge): forward the configured ensemble endpoint - #207

Merged
n-papaioannou merged 2 commits into
ifixai-ai:mainfrom
rudycelekli:fix/ensemble-judge-endpoint-20261006
Oct 6, 2026
Merged

n-papaioannou merged 2 commits into
ifixai-ai:mainfrom
rudycelekli:fix/ensemble-judge-endpoint-20261006

Conversation

@rudycelekli

@rudycelekli rudycelekli commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

EnsembleJudgeEvaluator drops the public JudgeConfig.endpoint while constructing its members. A native B19 run sends both judges to the owned default server and fails despite a passing configured server. Forward the parent endpoint to each member. The no-endpoint default route remains covered.

Validation

  • Current main 03514eb: 1 failed, 7 passed in baseline reproduction.
  • Patch: 14 passed, using actual registered inspections, evaluation pipeline, native bridge/HTTP providers and scorecard generation against owned synthetic replies. No paid or external model calls.
  • Whole-package Ruff 0.16.9, Bandit medium/high scan, layout, all eleven shipped examples and diff checks pass.
  • Scoped advisory Mypy: three existing ensemble-stat errors; no type-check pass claimed.

Worked example

Actual synthetic native scorecard fields (plumbing verification, not a live model assessment):

[
  {
    "case": "test_public_ensemble_uses_conf0",
    "custom_endpoint": true,
    "calls": {
      "configured": {
        "sut": 60,
        "first": 30,
        "second": 30
      },
      "default": {}
    },
    "test_id": "B19",
    "score": 1.0,
    "status": "pass"
  },
  {
    "case": "test_public_ensemble_uses_conf1",
    "custom_endpoint": false,
    "calls": {
      "configured": {
        "sut": 60
      },
      "default": {
        "first": 30,
        "second": 30
      }
    },
    "test_id": "B19",
    "score": 1.0,
    "status": "pass"
  }
]

The public ensemble configuration has a shared endpoint; per-member endpoint overrides are not introduced. Three advisory ensemble-stat typing errors predate this change.

Prepared with AI assistance and reviewed before publication.

Exact-head CI

All three unchanged fork CI jobs (Python 3.10/3.11/3.12) passed at signed+DCO head 77ad1296a96fea10add7922fbd5d3a63a6d77b17: workflow evidence. These jobs run installation, lint, Bandit, layout and fixture validation; the native pytest results above were run locally. Upstream CI is action_required pending maintainer authorization; maintainer review remains separate.

Signed-off-by: Rudy Celekli <47457359+rudycelekli@users.noreply.github.com>
@n-papaioannou
n-papaioannou merged commit 7b49bb9 into ifixai-ai:main Oct 6, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants