Repository navigation
fix(manifest): encode valid YAML metadata dates in digests - #214
n-papaioannou merged 4 commits into
Conversation
Signed-off-by: RudyCelekli <47457359+rudycelekli@users.noreply.github.com>
Signed-off-by: RudyCelekli <47457359+rudycelekli@users.noreply.github.com>
|
Thanks Rudy, dates as values work now, and every shipped fixture hashes the same as before. Dates as keys still crash: |
Signed-off-by: Rudy Celekli <rudy@gradiahq.com>
|
Confirmed: date keys failed before the JSON hashing step. The previous head failed three new cases; all nine current owners pass. Actual CLI runs with date and timestamp keys now write a scorecard and manifest using the owned mock provider. Ruff, Bandit, layout, all eleven fixtures and the digest module's advisory mypy pass. Current-head fork CI passes Python 3.10/3.11/3.12; upstream checks remain separate. |
What changed
A schema-valid YAML metadata date reaches
load_fixturebut crashesifixai runduring fixture digest creation, before the run can create its manifest. PyYAML's standard producer and loader use date/datetime objects for these scalars; existing TestCase.metadata admits them, and the fixture model's JSON export already writes ISO strings. Canonicalize these values as ISO strings compatible with the existing JSON-export contract. UTC timestamps may use +00:00 here versus Z in Pydantic JSON; this does not claim byte-identical datetime representations. Ordinary JSON-compatible fixture bytes remain unchanged.Native proof and validation
reviewed_onto an existing case; actual public validate exits0 and fixture load succeeds with date. Native public offline CLI then raises TypeError before the correction.Worked native example: the built-in mock SUT/judge run selects B01, writes one test result plus a manifest/report, and returns2 for its measured score. This is a real completed offline run; it does not establish provider efficacy or external model quality.
Verified SSH signature and DCO sign-off. AI assistance: Codex helped investigate, implement and test; complete source/tests and native evidence independently reviewed before publication.
October 7 maintainer follow-up
Signed forward repair
cf490ca2a17d2ec62359058d5b7b43e1357555dbalso canonicalises YAML date/timestamp keys before sorting. If an ISO conversion would merge two distinct keys, it fails explicitly rather than silently dropping an entry. Other key types retain the existing contract.Three new failures reproduce on the prior branch with six existing passing controls; all nine owner tests pass after the repair. Real public CLI subprocesses with date and timestamp metadata keys write one scorecard and one manifest each, using the owned mock provider. This establishes execution and hashing, not model quality. ISO-string equivalents have the same digest; ordinary fixture bytes retain their previous digest convention. Whole-package Ruff 0.15.21, Bandit, layout, eleven shipped example fixtures and the changed digest module's advisory mypy pass locally. Full inherited fork CI at this signed repair head passed all three Python 3.10/3.11/3.12 jobs (run 37702876446). Upstream checks are reported separately. This is maintenance of the existing PR, not a new contribution count.