Skip to content

DataSamplingBlock stratified sampling raises 'Sample larger than population' on valid inputs when topping up strata #15279

Description

@capy-ai

Summary

The stratified sampling method in DataSamplingBlock raises ValueError: Sample larger than population or is negative for valid inputs, because its top-up step adds samples to a stratum without checking how many items the stratum has.

Affected file and line

autogpt_platform/backend/backend/blocks/sampling.py: the stratum size adjustment loop (lines ~171-180, stratum_sizes[max(stratum_sizes, key=...)] += 1) followed by random.sample(strata[stratum], size) (line ~183).

Trigger and steps to reproduce

Run the block with sampling_method="stratified", stratify_key="g", sample_size=4 and data=[{"g":0},{"g":1},{"g":1},{"g":2},{"g":2}].

Initial sizes are 0 -> max(1, int(0.8)) = 1, 1 -> int(1.6) = 1, 2 -> 1, so the total is 3 and one more sample is needed. The loop increments the first stratum with the largest size (all tie at 1, so stratum 0), which has only 1 item, so random.sample([..1 item..], 2) raises.

Expected vs actual

  • Expected: 4 distinct items, with the extra sample going to a stratum that still has capacity (1 + 1 + 2, for example).
  • Actual: ValueError: Sample larger than population or is negative.

Severity

Low to medium. It fails loudly (no wrong data), but valid input crashes with a misleading error. A search over small datasets (n <= 5, up to 3 groups) finds failing cases quickly, so it is not an exotic input.

How I confirmed it

I ran DataSamplingBlock().run() directly over every small dataset up to 5 items with 3 groups; the first failing case is the one above. I did not look at the decrement branch in detail.

Fix feasibility

Yes: when topping up, pick a stratum where size < len(strata[k]), for example by choosing the largest remaining capacity.

Activity

  1. ntindle commented on Oct 8, 2026

    @ntindle
    Member

    Backfill validation: reproduced

    Image: significantgravitas/autogpt:latest @ sha256:122929723f57b8927af016d606ae21b21c03a012c2de1d9c30b1c8359e9157f1 (Hub v0.8.3 / sha-73cae306b4f6b197d2e1eaaa3162c326ec0ab076 / oci_revision 73cae306b4f6b197d2e1eaaa3162c326ec0ab076), pulled 2026-10-08 13:08 CT (ok_up_to_date), run sweep-20261008T1808.

    Verdict: reproduced (ran in-image backend code)

    What we checked

    sampling_method="stratified", stratify_key="g", sample_size=4 over [{g:0},{g:1},{g:1},{g:2},{g:2}] raises ValueError: Sample larger than population or is negative.

    Limits

    None beyond the reported case.

    Suggestion

    Top up only strata that still have spare capacity.

    Host evidence: /workspace/autogpt-backfill/evidence/15279/sweep-20261008T1808/ (host-local)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions