Fix C++ literal parser to match Python concore wire format#460
Open
GaneshPatil7517 wants to merge 2 commits intoControlCore-Project:devfrom
Open
Fix C++ literal parser to match Python concore wire format#460GaneshPatil7517 wants to merge 2 commits intoControlCore-Project:devfrom
GaneshPatil7517 wants to merge 2 commits intoControlCore-Project:devfrom
Conversation
- Replace stod-only parser with recursive descent parser in concore_base.hpp - Introduce ConcoreValue variant type supporting numbers, booleans, strings, nested arrays, and tuples (matching Python ast.literal_eval output) - Add parse_literal() and flatten_numeric() APIs to concore.hpp - Maintain full backward compatibility for flat numeric payloads - Add TestLiteralEvalCpp.cpp with 79 tests covering all payload types, error cases, and cross-language round-trip scenarios - Document wire format in README.md - Prevents silent cross-language data loss Fixes ControlCore-Project#389
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hey @pradeeban Sir
Fixes #389 The C++
parser()inconcore.hpppreviously usedstd::stod()on every comma-separated token, which silently crashed on valid concore payloads containing strings, booleans, nested lists, or tuples that Python nodes produce viaast.literal_eval().This PR replaces the stod-only parser with a minimal recursive descent parser that supports the full Python literal wire format, restoring cross-language compatibility.
Changes
concore_base.hppConcoreValuetype — A discriminated union (NUMBER,BOOL,STRING,ARRAY) mirroring Python'sast.literal_eval()outputparse_literal_value,parse_literal_string,parse_literal_array) handling:True/False→1.0/0.0)None(treated as string"None")flatten_numeric()— Recursively extracts all numeric values from aConcoreValueparselist_double()— Now delegates to the full parser; falls back gracefully on edge casesconcore.hppparse_literal()andflatten_numeric()methods on theConcoreclassparser()method unchanged in signature — full backward compatibilityTestLiteralEvalCpp.cpp(new)[10.0, "start", 0.5],[10.0, True, 0.5],[10.0, [0.5, 0.3], 0.1],(10.0, 0.3)ConcoreValuestructure verificationREADME.mdBefore / After
[10.0, 0.5][10.0, "start", 0.5]std::invalid_argument[10.0, 0.5](string skipped)[10.0, True, 0.5][10.0, 1.0, 0.5][10.0, [0.5, 0.3], 0.1][10.0, 0.5, 0.3, 0.1](10.0, 0.3)[10.0, 0.3]Testing
$ g++ -std=c++11 -o TestLiteralEvalCpp TestLiteralEvalCpp.cpp && ./TestLiteralEvalCpp
Results: 79 passed, 0 failed out of 79 tests
Per @pradeeban Sir instruction: this PR addresses the C++ implementation only. Verilog is not modified.