Skip to content

Conversation

@KRRT7
Copy link
Collaborator

@KRRT7 KRRT7 commented Feb 4, 2026

Summary

  • Add Babel-based function tracing for JavaScript/TypeScript projects
  • Captures function calls with nanosecond timing to SQLite database
  • Generates Jest/Vitest replay tests from traces
  • CLI integration with --language flag and auto-detection

Changes

  • JS Runtime: tracer.js, babel-tracer-plugin.js, trace-runner.js, replay.js
  • Python Integration: replay_test.py, tracer_runner.py, tracer.py
  • CLI: Language routing in main tracer.py
  • Refactor: Consolidated Jest/Vitest parsing into parse_test_output.py

Test plan

  • 21 unit and E2E tests in test_javascript_tracer.py
  • E2E test installs npm deps and runs full tracing pipeline
  • Added to js-tests.yml GitHub Actions workflow

@codeflash-ai
Copy link
Contributor

codeflash-ai bot commented Feb 4, 2026

⚡️ Codeflash found optimizations for this PR

📄 18% (0.18x) speedup for JavaScriptTracer._get_function_alias in codeflash/languages/javascript/tracer.py

⏱️ Runtime : 294 microseconds 250 microseconds (best of 250 runs)

A dependent PR with the suggested changes has been created. Please review:

If you approve, it will be merged into this PR (branch js-trace).

Static Badge

logger.info("Running JavaScript tracer: %s", " ".join(cmd))

try:
process = subprocess.run(cmd, cwd=project_root, env=env, capture_output=False, check=False)
Copy link

@claude claude bot Feb 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Still not fixed: The subprocess still runs with capture_output=False, which makes debugging failures very difficult. When the tracer fails (non-zero exit code), there's no stdout/stderr available to diagnose the issue.

Suggested fix:

process = subprocess.run(cmd, cwd=project_root, env=env, capture_output=True, text=True, check=False)

if process.returncode != 0:
    result["error"] = f"Tracing failed with exit code {process.returncode}\nStdout: {process.stdout}\nStderr: {process.stderr}"

conn.close()
return functions

except Exception:
Copy link

@claude claude bot Feb 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Still not fixed: Silent exception handling at line 296. Catching all exceptions without logging makes debugging failures impossible. If database parsing fails, users won't know why.

Suggested fix:

except Exception as e:
    logger.exception(f"Failed to get traced functions from {trace_file}: {e}")
    return []

sys.exit(1)

console.print(f"[green]Trace saved to: {result['trace_file']}[/green]")

Copy link

@claude claude bot Feb 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Acknowledged: The JavaScript tracer automatically triggers optimization without explicit user consent when not in trace-only mode. This is acceptable as it matches the intended workflow and is documented through the CLI flags.

from codeflash.languages.javascript.tracer import JavaScriptTracer


def node_available() -> bool:
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Consider adding helper: The E2E tests check for Node.js availability, but there's no helper to install npm dependencies. Consider adding a fixture that checks if node_modules/codeflash exists and provides setup instructions if missing.

@claude
Copy link

claude bot commented Feb 4, 2026

PR Review Summary

✅ Prek Checks

All prek checks (ruff format, ruff check) passed successfully. No auto-fixable issues found.

⚠️ MyPy Type Checking

Found 83 type errors across 4 files:

Critical issues in production code:

  • codeflash/languages/javascript/support.py (45 errors):

    • Missing type parameters for generic types (dict, list)
    • Type incompatibility issues (lines 1831-1832, 1705, 2212)
    • Operator type errors with Optional types (lines 1390, 1432)
  • codeflash/languages/javascript/tracer_runner.py (11 errors):

    • Missing return type annotations
    • Undefined attributes on subprocess results
  • codeflash/tracer.py (4 errors):

    • Import issues with Language from base module
    • Type incompatibility in tuple unpacking (line 168)

Test files (23 errors in test files - acceptable for test code):

  • Missing return type annotations (-> None)
  • Import issues with Language and FunctionParent

Recommendation: The production code type errors should be addressed in a follow-up PR, particularly the missing type parameters and Optional handling in support.py.

🔍 Code Review

Reviewed existing inline comments from previous review:

Outstanding Issues:

  1. ⚠️ tracer_runner.py:183 - Still using capture_output=False, making debugging failures difficult
  2. ⚠️ replay_test.py:296 - Silent exception handling without logging

Acknowledged:
3. ✅ tracer.py:348 - Auto-optimization behavior is acceptable as designed

No new critical issues found in the latest changes. The PR adds JavaScript tracing functionality with Babel instrumentation.

📊 Test Coverage

Changed Files Coverage:

File Coverage (PR) Coverage (Main) Change Status
replay_test.py (NEW) 64% N/A +64% ⚠️ Below 75%
tracer_runner.py (NEW) 34% N/A +34% 🔴 Below 75%
tracer.py 86% N/A +86% ✅ Good
support.py 74% 74% 0% ✅ Maintained
parse_test_output.py 42% 59% -17% ⚠️ Regression

Overall Coverage:

  • Main branch: 79.4%
  • PR branch: Unable to calculate due to deleted file in coverage data
  • Test results: 8 failing tests (same failures exist on main branch - not introduced by this PR)

Concerns:

  1. 🔴 tracer_runner.py: Only 34% coverage - critical new functionality is undertested
  2. ⚠️ replay_test.py: 64% coverage - below recommended 75% threshold
  3. ⚠️ parse_test_output.py: Coverage decreased from 59% to 42%

Recommendation: Add more unit tests for tracer_runner.py and replay_test.py to improve coverage above 75% threshold.


Last updated: 2026-02-05 17:51 UTC
Review by Claude Code Bot

@codeflash-ai
Copy link
Contributor

codeflash-ai bot commented Feb 4, 2026

⚡️ Codeflash found optimizations for this PR

📄 67% (0.67x) speedup for check_javascript_tracer_available in codeflash/languages/javascript/tracer_runner.py

⏱️ Runtime : 12.9 milliseconds 7.75 milliseconds (best of 124 runs)

A dependent PR with the suggested changes has been created. Please review:

If you approve, it will be merged into this PR (branch js-trace).

Static Badge

KRRT7 and others added 8 commits February 5, 2026 12:28
Add Babel-based function tracing for JavaScript/TypeScript:
- tracer.js: Core tracer with SQLite storage and nanosecond timing
- babel-tracer-plugin.js: AST transformation to wrap functions
- trace-runner.js: Entry point for running traced code
- replay.js: Utilities for replay test generation
- Update package.json with exports and dependencies
Add Python modules for JavaScript tracing orchestration:
- replay_test.py: Generate Jest/Vitest replay tests from traces
- tracer_runner.py: Run trace-runner.js and detect test frameworks
- tracer.py: Refactored to use Babel-only approach, removed legacy
  source transformation code
Move Jest/Vitest JUnit XML parsing from separate module into the main
parse_test_output.py file to reduce module fragmentation. Remove
duplicate tests that were testing the now-deleted module.
Add language detection and routing to JavaScript tracer in the main
tracer CLI. Supports --language flag and auto-detection from file
extensions in project.
Add comprehensive test coverage for JavaScript tracing:
- Unit tests for trace parsing (function_calls and legacy schemas)
- Unit tests for replay test generation (Jest and Vitest)
- E2E test for full tracing pipeline with npm dependencies
- Framework detection tests (Jest, Vitest, package.json)
- Add explicit type annotations for json.load return values

- Annotate result dict with dict[str, Any]

- Add type hints for config parameters

- Fix pytest_split unpacking to satisfy type checker

- Add null check for outfile before passing to run_javascript_tracer_main

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
}

console.log('[codeflash] Running Vitest with tracing...');
console.log('[codeflash] Note: ESM tracing requires additional setup. See documentation.');
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How are we implementing this as its quite common to have vitest with esm in current times.

const config = parseArgs(args);

// Register Babel with tracer plugin
registerBabel(config);
Copy link
Contributor

@Saga4 Saga4 Feb 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trying to understand how this will work when the transformation of codebases in vitest/webpack or esbuild happens before registering babel.
We might need more examples/tests with vitest and esm compatibility.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

workflow-modified This PR modifies GitHub Actions workflows

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants