Manual QA: test the pred CLI tool

[GiggleLiu]

Objective

Manually test the pred CLI tool end-to-end. This covers all subcommands, error handling, piping, and JSON output modes.

Setup

# Build the CLI in release mode
make cli

# Verify installation
./target/release/pred --version
./target/release/pred --help

Or use cargo run with dev profile:

alias pred='cargo run --manifest-path problemreductions-cli/Cargo.toml --'

Testing Checklist

1. Discovery commands

• pred list — lists all registered problems with aliases
• pred list --json — outputs valid JSON to stdout
• pred show MIS — shows variants, size fields, and reductions for MIS
• pred show MIS --json — JSON output
• pred show MIS/UnitDiskGraph — shows a specific graph variant
• pred to MIS — shows 1-hop outgoing neighbors
• pred to MIS --hops 2 — shows 2-hop outgoing neighbors
• pred from QUBO — shows 1-hop incoming neighbors
• pred from QUBO --hops 2 — shows 2-hop incoming neighbors

2. Path finding

• pred path MIS QUBO — finds cheapest path (minimize steps)
• pred path MIS QUBO --all — lists all available paths
• pred path MIS QUBO --cost minimize:num_variables — custom cost function
• pred path MIS QUBO -o path.json — saves path for use with pred reduce --via
• pred path MIS MIS — same source and target (edge case)
• pred path SAT QUBO — multi-step reduction path

3. Problem creation

• Graph problems:
  • pred create MIS --edges 0-1,1-2,2-3 -o problem.json
  • pred create MIS --edges 0-1,1-2 --weights 2,1,3
  • pred create MVC --edges 0-1,1-2,2-0
  • pred create MaxCut --edges 0-1,1-2,2-3
• SAT problems:
  • pred create SAT --num-vars 3 --clauses "1,2;-1,3"
  • pred create 3SAT --num-vars 4 --clauses "1,2,3;-1,-2,4"
• QUBO:
  • pred create QUBO --matrix "1,0.5;0.5,2"
• KColoring:
  • pred create KColoring --k 3 --edges 0-1,1-2,2-0
• Factoring:
  • pred create Factoring --target 15 --bits-m 4 --bits-n 4
• Random generation:
  • pred create MIS --random --num-vertices 10 --edge-prob 0.3
  • pred create MIS --random --num-vertices 10 --seed 42 — reproducible (run twice, compare)

4. Evaluate

• pred evaluate problem.json --config 1,0,1,0 — evaluates a config
• pred create MIS --edges 0-1,1-2 | pred evaluate - --config 1,0,1 — piping from stdin
• Test invalid config length (should error gracefully)

5. Reduce

• pred reduce problem.json --to QUBO -o reduced.json — direct reduction
• pred reduce problem.json --via path.json -o reduced.json — via saved path
• pred create MIS --edges 0-1,1-2 | pred reduce - --to QUBO — piping
• pred inspect reduced.json — inspect the reduction bundle

6. Solve

• pred solve problem.json — ILP solver (default)
• pred solve problem.json --solver brute-force — brute-force solver
• pred solve reduced.json — solve a reduction bundle (solution mapped back)
• pred create MIS --edges 0-1,1-2 | pred solve - — piping
• pred solve problem.json --timeout 10 — timeout option
• Compare ILP and brute-force results on the same instance — should agree

7. Full closed-loop workflow

Test the complete pipeline end-to-end:

# Create → Solve → Verify
pred create MIS --edges 0-1,1-2,2-3,3-0 -o problem.json
pred solve problem.json -o result.json
# Check that the solution config is valid:
pred evaluate problem.json --config <solution_from_result>

# Create → Reduce → Solve → Verify
pred create MIS --edges 0-1,1-2,2-3 -o problem.json
pred path MIS QUBO -o path.json
pred reduce problem.json --via path.json -o reduced.json
pred solve reduced.json -o result.json
# Solution should be mapped back to MIS space

# Full pipe chain
pred create MIS --edges 0-1,1-2,2-3 | pred reduce - --to QUBO | pred solve -

8. JSON output and piping

• Every command with --json produces valid JSON (pipe through jq . to verify)
• Every command with -o file.json writes valid JSON to the file
• Stdin reading with - works for: evaluate, reduce, solve, inspect
• --quiet suppresses stderr info messages while still producing stdout output

9. Error handling

• Unknown problem name → clear error message with suggestions
• Missing required args (e.g., pred create MIS without --edges) → helpful error
• Invalid edge format → clear error
• Non-existent reduction path → informative message
• Invalid JSON input file → graceful error
• Mismatched config length in pred evaluate → clear error

10. Shell completions

• eval "$(pred completions)" — installs completions without error
• Tab completion works for subcommands and problem names

Reporting

For each failed item, please note:

1. The exact command run
2. The actual output / error
3. The expected behavior

File findings as comments on this issue or as separate bug issues referencing this one.

[GiggleLiu]

@isPANN These test steps requires human input. It would be great if you could help.