Skip to content

Refactor documentation examples to doctests for continuous validation #2106

@Borda

Description

@Borda

Search before asking

  • I have searched the issues and discussions and found no similar feature request.

🚀 The Problem

Currently, many documentation examples within the codebase are written as static Markdown code blocks. While these serve as visual guides, they are not executed or verified by the CI/CD pipeline.

Because these examples are not part of the test suite:

  1. Code Rot: As the API evolves/refactors, documentation examples frequently become outdated or incorrect without anyone noticing.
  2. User Friction: Users copying "official" examples may encounter immediate errors, leading to confusion and a lack of trust in the docs.
  3. Missed Coverage: These examples often represent the primary "happy path" for users, yet they are currently excluded from regression testing.

💡 Proposed Solution

We should refactor these valid usage examples into Python's standard doctest format. This allows them to be executed and verified automatically by the test runner (e.g., via pytest --doctest-modules).

🏆 Justification (Industry Standard)

Adopting doctest is a widely accepted best practice in the Python ecosystem (standard in libraries like Pandas, NumPy, and Scikit-Learn).

  • Verification: It ensures the documentation is a test. If the docs say it works, the CI proves it.
  • Maintenance: It catches breaking changes in the public API immediately during PR checks.
  • User Experience: It guarantees that the first piece of code a user tries will work as expected.

Example Usage

Current State (Untested static block)

  '''python
  import cv2
  import supervision as sv
  from ultralytics import YOLO
  
  image = cv2.imread(...)
  model = YOLO('yolov8s.pt')
  result = model(image)[0]
  detections = sv.Detections.from_ultralytics(result)
  '''

Desired State (Tested doctest)

  >>> import cv2
  >>> import supervision as sv
  >>> from ultralytics import YOLO
  >>> image = cv2.imread("tests/data/image.jpg")
  >>> model = YOLO('yolov8s.pt')
  >>> result = model(image)[0]
  >>> detections = sv.Detections.from_ultralytics(result)
  >>> len(detections) > 0  # verifies the code actually ran and produced output
  True

Are you willing to submit a PR?

  • Yes I'd like to help by submitting a PR!

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestopen-to-multiple-contributorsMultiple contributors can work on this issue, but typically, only one submission will be accepted.

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions