Skip to content

Ensure example_id exists in init_state (default to 0)#990

Closed
willccbb wants to merge 1 commit intomainfrom
codex/add-example_id-to-init_state
Closed

Ensure example_id exists in init_state (default to 0)#990
willccbb wants to merge 1 commit intomainfrom
codex/add-example_id-to-init_state

Conversation

@willccbb
Copy link
Member

@willccbb willccbb commented Mar 5, 2026

Motivation

  • Prevent missing example_id from causing downstream errors by ensuring init_state always supplies a valid example_id when inputs omit it.

Description

  • Add a guard in Environment.init_state to set state_input["example_id"] = 0 when example_id is not present before constructing the State (file: verifiers/envs/environment.py).

Testing

  • Ran uv run pytest tests/test_environment.py -k init_state which passed (2 passed, 28 deselected).
  • Ran uv run ruff check verifiers/envs/environment.py which completed with all checks passing.

Codex Task


Note

Low Risk
Small, localized defaulting change in state initialization; main risk is minor behavior change for callers that relied on absent example_id.

Overview
Ensures Environment.init_state always populates state_input["example_id"] by defaulting to 0 when the input row omits it, preventing downstream code from failing on missing IDs.

Written by Cursor Bugbot for commit d74f297. This will update automatically on new commits. Configure here.

Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

if "info" in state_input and isinstance(state_input["info"], str):
state_input["info"] = json.loads(state_input["info"])
if "example_id" not in state_input:
state_input["example_id"] = 0
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All missing example_ids silently collapse to same value

Medium Severity

When example_id is missing from the input, it defaults to the constant 0 for every item. If init_state is called for multiple inputs that all lack example_id (e.g., when bypassing the dataset pipeline that assigns unique IDs via _ensure_example_id), all those items silently share example_id = 0. This can cause incorrect grouping or tracking downstream, since example_id is expected to uniquely identify examples — as emphasized by _ensure_example_id and EnvGroup logic that explicitly ensures uniqueness. A per-call unique value or an incrementing counter would avoid silent collisions.

Fix in Cursor Fix in Web

if "info" in state_input and isinstance(state_input["info"], str):
state_input["info"] = json.loads(state_input["info"])
if "example_id" not in state_input:
state_input["example_id"] = 0
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing documentation update for new default behavior

Low Severity

This PR modifies the behavior of init_state — a method documented in docs/reference.md (line 306) — by making example_id effectively optional with a default of 0. The docs currently list example_id as "Required" in RolloutInput (line 120). Per the documentation update rule, changes to core user-facing functionality described in docs/ need corresponding documentation updates to reflect the new defaulting behavior.

Fix in Cursor Fix in Web

Triggered by project rule: BugBot Instructions

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant