Skip to content

test: Add cross runtime protocol conformance matrix (Phase 2)#467

Open
Titas-Ghosh wants to merge 1 commit intoControlCore-Project:devfrom
Titas-Ghosh:add-protocol-conformance-phase2-mapping
Open

test: Add cross runtime protocol conformance matrix (Phase 2)#467
Titas-Ghosh wants to merge 1 commit intoControlCore-Project:devfrom
Titas-Ghosh:add-protocol-conformance-phase2-mapping

Conversation

@Titas-Ghosh
Copy link

Hi @pradeeban,
Following up on the discussion in #440 and Phase 1 conformance work, here is the PR for Phase 2 (mapping/audit).

As promised in our initial plan, this PR is strictly report only and non breaking. It introduces a language agnostic matrix to track how different runtimes handle the protocol cases we defined in Phase 1, without changing a single line of runtime logic anywhere.

What Changed

  • The Matrix (cross_runtime_matrix.phase2.json): Added phase 2 metadata (phase, mode, runtimes, classifications, statuses).
    • Mapped each Phase 1 case across python, cpp, matlab, octave, and verilog.
    • Crucially, all non Python runtimes are explicitly marked as not_audited to ensure this phase remains purely observational.
  • The Runner (tests/test_protocol_conformance_phase2.py):
    • Validates the matrix metadata and enums.
    • Ensures a 1:1 mapping with Phase 1 case IDs.
    • Enforces row shape consistency and verifies the python=observed_pass baseline.
  • Documentation (PROTOCOL_FIXTURES.md): Updated to clearly document the Phase 2 scope and constraints.

Safety

  • Zero production/runtime files were modified. There are absolutely no behavior changes for any existing language bindings.
  • We are not forcing cross-language behavior; this just adds the structured audit foundation so we can review the landscape before considering any future enforcement.

Validation

  • Targeted conformance tests:
    python -m pytest -v tests/test_protocol_conformance.py tests/test_protocol_conformance_phase2.py10 passed
  • Broader non-regression run:
    python -m pytest -v tests --ignore=tests/test_openjupyter_security.py99 passed

Evidence

1) Phase-2 matrix metadata (report only + enums)

Screenshot 2026-02-25 011743

2) Example mapped case (Python observed, others pending)

Screenshot 2026-02-25 013418

3) Targeted conformance tests passing

Screenshot 2026-02-25 011859

4) Broader test run passing

Screenshot 2026-02-25 011936

Note to Maintainers

I kept this intentionally minimal and report only so we can review the matrix structure together before we start classifying how the other languages currently behave. let me know if this structure looks good to you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant