Description
Problem
predict(ids=...) is currently rejected when nonlocal pooled lag transforms are present. This is correct for pure global transforms, because they pool across all series. However, it is overly restrictive for grouped pooled transforms.
For example, with RollingMean(..., groupby=["store_id"]), the transform only pools series within each store_id. Predicting a subset should be safe when the selected ids include every series in each selected store.
Desired Behavior
Allow subset prediction when the selected series preserve the pooled dependencies required by every lag transform.
The invariant should be:
A selected subset is valid only if every non-global pooled state can be evaluated without dropping part of any pooled group needed by the selected forecast population.
This applies across multiple grouped configurations simultaneously. If transforms use both groupby=["store_id"] and groupby=["region"], the selected subset must form complete groups for both groupings.
Partition By Compatibility
The feature/pooled_partition_by branch generalizes pooled state via _pooled_states keyed as (mode, group_cols, partition_cols). The validation should target that abstraction rather than special-casing only _pooled_groups.
Suggested rules:
| Pooled state |
Example key |
Subset behavior |
| Pure global |
("global", (), ()) |
Require all ids |
| Pure groupby |
("groupby", ("brand",), ()) |
Allow complete selected groups |
| Local partition |
("local", (), ("promo",)) |
Allow arbitrary ids |
| Global + partition |
("nonlocal", (), ("promo",)) |
Require all ids |
| Groupby + partition |
("nonlocal", ("brand",), ("promo",)) |
Allow complete selected groups |
partition_by values do not need their own filter parameter. They are resolved from X_df or static features at prediction time. The subset safety condition is about preserving group completeness for the parent pooled population, not selecting partition values.
API Options
Option A: Infer From ids
Keep the current public API:
predict(..., ids=[...])
When ids is provided:
- reject if any pure global or global + partition pooled state is present;
- allow local partition states;
- for groupby and groupby + partition states, verify that selected ids form complete groups for every distinct
group_cols configuration;
- raise a clear error if any grouping is partially selected.
This is the smallest API change and preserves current usage.
Option B: Add groupby_values
Add an optional group-based selector, for example:
predict(..., groupby_values={"store_id": ["A", "B"]})
This would select all ids belonging to the requested group values, then run the same pooled-state closure validation as Option A.
Important details:
groupby_values should select ids only through groupby columns, not partition_by columns.
- It should still validate closure across all pooled states.
- If transforms use multiple groupby configurations, selecting values for one grouping may still be invalid if it partially selects another grouping.
- If both
ids and groupby_values are supplied, either reject the combination or require their implied id sets to match exactly.
This gives users a clearer way to request complete groups without manually expanding group values into ids.
Recommended Direction
Implement Option A first, because it requires no new public API and fixes the current overly restrictive behavior.
Option B can be added as a convenience layer later. It should compile groupby_values into an id set and reuse the same pooled-state closure validator.
Implementation Notes
The existing _maybe_subset path already temporarily subsets ga, uids, static features, dates, target transforms, and lag transforms. It should also temporarily subset pooled state.
For grouped and grouped + partition states:
- use
series_bucket_id to map ids to pooled buckets;
- check that selected ids include all series for every selected bucket;
- subset pooled arrays and metadata to selected buckets;
- compact bucket ids so temporary state remains internally consistent;
- restore the original pooled state after prediction.
For local partition_by, arbitrary id subsets are semantically safe, but the pooled state still needs temporary subsetting or remapping so state.series_bucket_id matches the narrowed self.uids.
Tests
Add pandas and polars coverage for:
ids with pure global transforms still raises.
ids with one groupby transform succeeds for complete groups.
ids with one groupby transform raises for partial groups.
- multiple
groupby configurations require closure across all groupings.
- local
partition_by allows arbitrary id subsets.
- global +
partition_by still requires all ids.
- groupby +
partition_by allows complete selected groups.
groupby_values selects complete groups if that API is added.
groupby_values still fails when it breaks another groupby configuration.
- subset predictions match full predictions filtered to the same ids.
- prediction state is restored after successful or failed subset prediction.
Use case
I don't have a use case myself for now but it might be good to allow this as it is overly restrictive at this stage
Description
Problem
predict(ids=...)is currently rejected when nonlocal pooled lag transforms are present. This is correct for pure global transforms, because they pool across all series. However, it is overly restrictive for grouped pooled transforms.For example, with
RollingMean(..., groupby=["store_id"]), the transform only pools series within eachstore_id. Predicting a subset should be safe when the selected ids include every series in each selected store.Desired Behavior
Allow subset prediction when the selected series preserve the pooled dependencies required by every lag transform.
The invariant should be:
This applies across multiple grouped configurations simultaneously. If transforms use both
groupby=["store_id"]andgroupby=["region"], the selected subset must form complete groups for both groupings.Partition By Compatibility
The
feature/pooled_partition_bybranch generalizes pooled state via_pooled_stateskeyed as(mode, group_cols, partition_cols). The validation should target that abstraction rather than special-casing only_pooled_groups.Suggested rules:
("global", (), ())("groupby", ("brand",), ())("local", (), ("promo",))("nonlocal", (), ("promo",))("nonlocal", ("brand",), ("promo",))partition_byvalues do not need their own filter parameter. They are resolved fromX_dfor static features at prediction time. The subset safety condition is about preserving group completeness for the parent pooled population, not selecting partition values.API Options
Option A: Infer From
idsKeep the current public API:
predict(..., ids=[...])When
idsis provided:group_colsconfiguration;This is the smallest API change and preserves current usage.
Option B: Add
groupby_valuesAdd an optional group-based selector, for example:
predict(..., groupby_values={"store_id": ["A", "B"]})This would select all ids belonging to the requested group values, then run the same pooled-state closure validation as Option A.
Important details:
groupby_valuesshould select ids only throughgroupbycolumns, notpartition_bycolumns.idsandgroupby_valuesare supplied, either reject the combination or require their implied id sets to match exactly.This gives users a clearer way to request complete groups without manually expanding group values into ids.
Recommended Direction
Implement Option A first, because it requires no new public API and fixes the current overly restrictive behavior.
Option B can be added as a convenience layer later. It should compile
groupby_valuesinto an id set and reuse the same pooled-state closure validator.Implementation Notes
The existing
_maybe_subsetpath already temporarily subsetsga,uids, static features, dates, target transforms, and lag transforms. It should also temporarily subset pooled state.For grouped and grouped + partition states:
series_bucket_idto map ids to pooled buckets;For local
partition_by, arbitrary id subsets are semantically safe, but the pooled state still needs temporary subsetting or remapping sostate.series_bucket_idmatches the narrowedself.uids.Tests
Add pandas and polars coverage for:
idswith pure global transforms still raises.idswith onegroupbytransform succeeds for complete groups.idswith onegroupbytransform raises for partial groups.groupbyconfigurations require closure across all groupings.partition_byallows arbitrary id subsets.partition_bystill requires all ids.partition_byallows complete selected groups.groupby_valuesselects complete groups if that API is added.groupby_valuesstill fails when it breaks another groupby configuration.Use case
I don't have a use case myself for now but it might be good to allow this as it is overly restrictive at this stage