Claude/claude md mmjvy980os5wciap y v qol#2671
Claude/claude md mmjvy980os5wciap y v qol#2671tommykho wants to merge 3 commits intopytorch:mainfrom
Conversation
Documents repository structure, development workflows, key conventions, model/userbenchmark APIs, CI/CD overview, and common patterns for the PyTorch benchmark suite. https://claude.ai/code/session_013qCnFwRGCxLCgZRgsSHyNY
Implements a new userbenchmark at userbenchmark/whisper/ that benchmarks the OpenAI Whisper medium encoder on both CPU (fp32) and CUDA (fp16). Measures: - Per-inference latency: warmup loop + timed loop using time.time_ns() with torch.cuda.synchronize() bracketing, following the pattern in torchbenchmark/util/experiment/metrics.py::get_latencies() - Encoder FLOPs via torch.utils.flop_counter.FlopCounterMode, matching the approach in torchbenchmark/util/experiment/metrics.py::get_model_flops() - CPU-to-GPU speedup ratio (when CUDA is available) Input shape (1, 80, 3000) matches the existing hf_Whisper torchbenchmark model. CUDA inference uses fp16 to match hf_Whisper's DEFAULT_EVAL_CUDA_PRECISION = "fp16". Outputs a human-readable table and writes a userbenchmark JSON file to .userbenchmark/whisper/metrics-<timestamp>.json. Usage: python run_benchmark.py whisper python run_benchmark.py whisper --no-cuda python run_benchmark.py whisper --warmup 3 --iters 5 --no-flops python userbenchmark/whisper/run.py # direct execution https://claude.ai/code/session_013qCnFwRGCxLCgZRgsSHyNY
`exec(f"from transformers import {config_cls_name}")` does not make the
imported name available to a subsequent `eval()` call in the same function
scope in Python 3. Pass an explicit namespace dict to both `exec` and `eval`
so the imported class is accessible.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
Hi @tommykho! Thank you for your pull request and welcome to our community. Action RequiredIn order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you. ProcessIn order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA. Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with If you have received this in error or have any questions, please contact us at cla@meta.com. Thanks! |
2 participants
No description provided.