Performance: ByteTables + targeted byte-walking for hot parse paths

[cpakman]

Summary

Add pre-computed byte lookup tables (ByteTables) and apply byte-walking to three hot parsing paths, replacing regex matching and StringScanner usage in the most frequently called parse methods.

+208 / -61 across 5 files (net +147 lines of production code).

Results (Ruby 4.0.2 + YJIT, theme benchmark)

Metric	main	this branch	Δ
Parse	6,426µs	5,529µs	-14%
Render	1,557µs	1,516µs	-3%
Combined	7,983µs	7,045µs	-11%
Allocations	60,811	51,620	-15%

What changed

1. ByteTables module (new, 44 lines)

Pre-computed 256-entry boolean arrays for byte classification: IDENT_START, IDENT_CONT, WORD, DIGIT, WHITESPACE. A single array index (TABLE[byte]) replaces 3-5 chained comparisons per byte check. Built once at load time, frozen.

2. Expression.parse_number — byte-walk instead of regex + StringScanner

Replaced INTEGER_REGEX/FLOAT_REGEX matching and the StringScanner byte loop with a single forward pass using ByteTables::DIGIT. Avoids MatchData allocation and StringScanner reset per call. Handles all edge cases: negative numbers, multi-dot floats (1.2.3.4), trailing dots (123.), and rejects trailing non-numeric bytes (1.2.3a).

Also guards Expression.parse's String#strip call — only allocates when whitespace is actually present (~4,464 avoided allocations per compile).

3. VariableLookup — fast path for simple identifier chains

SIMPLE_LOOKUP_RE validates that input is a plain a.b.c chain (no brackets, no quotes — ~90% of real-world lookups). On match, byte-walks on dots instead of invoking the recursive VariableParser regex (/\[(?>[^\[\]]+|\g<0>)*\]|[\w-]+\??/). Falls through to original path for complex inputs.

4. BlockBody.try_parse_tag_token — byte-walk tag tokens

Parses {%...%} tokens using getbyte/byteslice + ByteTables instead of the FullToken regex with 4 capture groups. Allocates only the 2 strings needed (tag_name, markup) vs 4+ from regex captures. Uses ByteTables::WORD (no hyphen) for tag name scanning, matching TagName = /#|\w+/ exactly. Falls back to FullToken regex on nil.

Design principles

• Every fast path has a regex fallback — if the byte-walker returns nil/false, the original regex path runs. Zero risk for edge cases.
• No new abstractions — no Cursor class, no parallel parse pipeline. Just targeted byte-walking at three specific call sites.
• Each commit is independently revertable — one file per optimization commit.

Review process

This was developed iteratively with multi-agent code review covering:

• Correctness: Found and fixed 3 issues (hyphen in tag names via IDENT_CONT, ? suffix in tag names, multi-dot trailing alpha in parse_number)
• Security: Found and fixed null byte gap in WHITESPACE table vs String#strip
• Performance: Confirmed match? + byte-walk is near-optimal (pure byte-walk is 2.8× slower than regex for validation)
• Simplicity: Named constants, structural comments, consistent patterns across all three sites
• Rubocop: 0 offenses