Add regression test for deterministic equal-priority task order#4666
Add regression test for deterministic equal-priority task order#4666davidahmann wants to merge 2 commits intocrewAIInc:mainfrom
Conversation
|
Implemented the scoped fix for #4664 by adding deterministic equal-priority ordering regression coverage for crew task execution ordering. Validation run:
Current CI snapshot: is still pending. This contribution was informed by patterns from Wrkr. Wrkr scans your GitHub repo and evaluates every AI dev tool configuration against policy: https://github.com/Clyra-AI/wrkr |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e683d6b06a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
lib/crewai/tests/test_crew.py
Outdated
| description="Task B", | ||
| expected_output="Output B", | ||
| agent=researcher, | ||
| config={"priority": 1}, |
There was a problem hiding this comment.
Exercise actual priority field in ordering regression test
This test is intended to validate equal-priority ordering, but Task drops unknown config keys during model validation (only keys present in Task.model_fields are copied, then config is removed), and priority is not a Task field. In practice these tasks are executed as ordinary sequential tasks with no priority metadata, so the assertion can still pass even if equal-priority scheduling behavior regresses.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Bugbot Free Tier Details
Your team is on the Bugbot Free tier. On this plan, Bugbot will review limited PRs each billing cycle for each member of your team.
To receive Bugbot reviews on all of your PRs, visit the Cursor dashboard to activate Pro and start your 14-day free trial.
Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Problem
Equal-priority task execution ordering was not explicitly regression-tested, risking non-deterministic sequencing across runs.
Why now
Deterministic crew/task lifecycle behavior is required for stable orchestration and replayability.
What changed
test_equal_priority_tasks_preserve_declared_orderinlib/crewai/tests/test_crew.py.Agent.execute_task.tasks_outputorder match declaration order.Validation
uv run pytest -q tests/test_crew.py -k equal_priority_tasks_preserve_declared_orderRefs #4664
Note
Low Risk
Low risk: test-only change that adds coverage for deterministic task ordering without modifying runtime code paths.
Overview
Adds a new regression test ensuring sequential crews execute tasks in the exact order they are declared, even when task descriptions could sort differently (e.g.,
Task 2,Task 10,Task 1).The test patches
Agent.execute_taskto capture the observed execution sequence and asserts both execution order andCrewOutput.tasks_outputordering match the declared task list.Written by Cursor Bugbot for commit 2caac65. This will update automatically on new commits. Configure here.