Repository navigation
Conversation
A choice narrowed by predict_shortlist or predict_tournament kept the caller's full-length option_order, which Agent._check_question then refuses: the order must be a permutation of range(n_options) and the kept label set is smaller. Nandha called this a core contract question on NandhaKishorM#958 and asked for the order to be dropped with a warning. These checks are red on main; they pin the narrowed question validating, one RuntimeWarning naming the question, an answer identical to the no-order call, and passthrough/non-choice questions untouched.
predict_shortlist narrowed criteria to the top-k labels while forwarding the caller's full-length option_order, and predict_tournament built its group and finalist questions the same way, so a choice carrying an order crashed in the agent: the order must be a permutation of range(n_options) and the kept label set is smaller. Nandha called this a core contract question on NandhaKishorM#958 and asked for the order to be dropped with a warning, since remapping it onto the kept labels would silently move options between slots. Both paths now drop the order on a narrowed question and emit one RuntimeWarning naming the question and the slot counts. Passthrough questions, non-choice questions and the caller's own dicts are untouched, so a call with an order answers exactly like the same call without one.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
predict_shortlistnarrows a choice'scriteriato the top-k labels while forwarding the caller's full-lengthoption_order, andpredict_tournamentbuilds its group and finalist questions the same way incut(). The narrowed question then failsAgent._check_question, because the order must be a permutation ofrange(n_options)and the kept label set is smaller. Reproduced on main1adc59f7:Fix
Following the direction you gave on #958, a narrowed choice now drops its
option_orderwith oneRuntimeWarningnaming the question, the old slot count and the kept label count. The order is not remapped onto the kept labels, since that would silently move options between slots.Both paths are covered:
predict_shortlist, andcut()inpredict_tournament(the round groups and the finalists share that choke point, so one helper covers them). The warning fires once per question per call, not once per tournament round.Unchanged: passthrough questions (n <= k), non-choice questions, the caller's own dicts (both paths copy before editing), and calls without an
option_order— those emit no warning and behave as before, so a call with an order answers exactly like the same call without one. Verified on real weights for both paths.Tests
python tests/test_shortlist.py→ 219 passed, 0 failed (on main: 209 passed, 10 failed — the 10 new checks)python tests/test_hooks_api.py→ 572 passed, 0 failedruff check laya/ --select=E9,F63,F7,F82,F401,F811 --line-length=120clean;python -m compileall -q laya/ tests/exit 0tests/test_mcp.pycould not run in my environment: the installedmcpis 1.29.1 and the suite requires >= 2.2.0. It fails the same way on unmodified main, so this is an environment block, not a result — and MCP behaviour is unchanged, since fix(mcp): keep and validate a question's option_order #958 already makeslaya_shortlistrefuse a narrowed question that carries an order.Two commits: the regression tests first (red on main, green with the fix), then the fix itself.
Closes #1062