Skip to content

Fix C++ literal parser to match Python concore wire format#460

Open
GaneshPatil7517 wants to merge 2 commits intoControlCore-Project:devfrom
GaneshPatil7517:fix/cpp-literal-parser
Open

Fix C++ literal parser to match Python concore wire format#460
GaneshPatil7517 wants to merge 2 commits intoControlCore-Project:devfrom
GaneshPatil7517:fix/cpp-literal-parser

Conversation

@GaneshPatil7517
Copy link

@GaneshPatil7517 GaneshPatil7517 commented Feb 24, 2026

Hey @pradeeban Sir
Fixes #389 The C++ parser() in concore.hpp previously used std::stod() on every comma-separated token, which silently crashed on valid concore payloads containing strings, booleans, nested lists, or tuples that Python nodes produce via ast.literal_eval().

This PR replaces the stod-only parser with a minimal recursive descent parser that supports the full Python literal wire format, restoring cross-language compatibility.

Changes

concore_base.hpp

  • ConcoreValue type — A discriminated union (NUMBER, BOOL, STRING, ARRAY) mirroring Python's ast.literal_eval() output
  • Recursive descent parser (parse_literal_value, parse_literal_string, parse_literal_array) handling:
    • Numbers (int, float, negative, scientific notation)
    • Booleans (True/False1.0/0.0)
    • Strings (single/double quoted with escape sequences)
    • Nested arrays and tuples (treated identically)
    • None (treated as string "None")
    • Trailing commas
  • flatten_numeric() — Recursively extracts all numeric values from a ConcoreValue
  • parselist_double() — Now delegates to the full parser; falls back gracefully on edge cases

concore.hpp

  • Exposed parse_literal() and flatten_numeric() methods on the Concore class
  • Existing parser() method unchanged in signature — full backward compatibility

TestLiteralEvalCpp.cpp (new)

  • 79 tests covering:
    • Backward compatibility (flat numeric, empty, negative, scientific notation)
    • Mixed-type payloads: [10.0, "start", 0.5], [10.0, True, 0.5], [10.0, [0.5, 0.3], 0.1], (10.0, 0.3)
    • Full ConcoreValue structure verification
    • Error handling (malformed brackets, unterminated strings, unsupported objects)
    • Cross-language round-trip (Python write → C++ read)

README.md

  • Added Wire Format section documenting the concore payload specification

Before / After

Payload Before (C++) After (C++)
[10.0, 0.5] ✅ Works ✅ Works
[10.0, "start", 0.5] std::invalid_argument [10.0, 0.5] (string skipped)
[10.0, True, 0.5] ❌ Crash [10.0, 1.0, 0.5]
[10.0, [0.5, 0.3], 0.1] ❌ Crash [10.0, 0.5, 0.3, 0.1]
(10.0, 0.3) ❌ Empty [10.0, 0.3]

Testing

$ g++ -std=c++11 -o TestLiteralEvalCpp TestLiteralEvalCpp.cpp && ./TestLiteralEvalCpp
Results: 79 passed, 0 failed out of 79 tests

Per @pradeeban Sir instruction: this PR addresses the C++ implementation only. Verilog is not modified.

- Replace stod-only parser with recursive descent parser in concore_base.hpp
- Introduce ConcoreValue variant type supporting numbers, booleans, strings,
  nested arrays, and tuples (matching Python ast.literal_eval output)
- Add parse_literal() and flatten_numeric() APIs to concore.hpp
- Maintain full backward compatibility for flat numeric payloads
- Add TestLiteralEvalCpp.cpp with 79 tests covering all payload types,
  error cases, and cross-language round-trip scenarios
- Document wire format in README.md
- Prevents silent cross-language data loss

Fixes ControlCore-Project#389
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant