Optimize webhooks JSON with compact structure and string interning#232
Optimize webhooks JSON with compact structure and string interning#232ericsciple wants to merge 1 commit intomainfrom
Conversation
bfb514f to
4dacd2c
Compare
4dacd2c to
27aec5d
Compare
| run: cd languageservice && npm test -- --testPathPattern=eventPayloads | ||
| - name: Verify validation tests ran | ||
| run: | | ||
| if [ ! -f languageservice/src/context-providers/events/webhooks.full.validation-complete ]; then |
There was a problem hiding this comment.
Since the test is normally skipped when the full webhooks file doesn't exist, I made the test write a marker file. With the marker file, we can be sure the test actually ran.
The full webhooks file (unoptimized) is written above by the script npm run update-webhooks
| exit 1 | ||
| fi | ||
|
|
||
| validate-webhooks: |
There was a problem hiding this comment.
I added this job to validate the optimized webhooks JSON is equivalent to the full webhooks JSON file.
Refer changes to the file eventPayloads.test.ts in this PR
27aec5d to
8fbd093
Compare
| @@ -1,310 +0,0 @@ | |||
| import {promises as fs} from "fs"; | |||
There was a problem hiding this comment.
I renamed this file to update-webhooks.ts but there were also significant updates anyway
| @@ -1,7 +1,8 @@ | |||
| import {data, DescriptionDictionary} from "@actions/expressions"; | |||
|
|
|||
There was a problem hiding this comment.
eventsPayloads.ts reads the compact webhook JSON files which contains the keys and descriptions for each GitHub event payload. This information is used for github.event.* syntax validation, autocompletion, and hover descriptions.
A unit test that runs during CI validates the compact webhook JSON files are parsed into a form that exactly matches the full unoptimized webhooks JSON.
The script update-webhooks.ts downloads the full webhook JSON from the GitHub REST API description repository:
import schemaImport from "rest-api-description/descriptions/api.github.com/dereferenced/api.github.com.deref.json";10dd69f to
2b6d46c
Compare
There was a problem hiding this comment.
Pull request overview
This PR optimizes webhook payload JSON files used for GitHub Actions autocompletion and hover documentation by introducing a compact array format, object deduplication, and string interning. The optimizations achieve a 55% reduction in minified file size (464 KB → 209 KB) and 65% reduction when gzipped (63 KB → 22 KB).
Key Changes:
- Introduced compact array format for params using type-based dispatch instead of verbose objects
- Implemented string interning to deduplicate 414 property names into a shared
strings.jsonfile - Extracted event filtering configuration from TypeScript to
event-filters.jsonfor use in both generation and test code
Reviewed changes
Copilot reviewed 11 out of 14 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
languageservice/src/context-providers/events/strings.json |
New string table containing 414 interned property names referenced by index |
languageservice/src/context-providers/events/eventPayloads.ts |
Updated to load separate string table and decode compact format with string interning support |
languageservice/src/context-providers/events/eventPayloads.test.ts |
Added comprehensive validation tests comparing optimized output against full source |
languageservice/src/context-providers/events/event-filters.json |
New JSON file defining dropped/kept events, replacing hardcoded TypeScript constants |
languageservice/script/webhooks/update-webhooks.ts |
New generation script implementing compact format conversion, deduplication, and string interning |
languageservice/script/webhooks/event-filters.ts |
TypeScript constants for event filtering (note: appears unused, actual imports use JSON file) |
languageservice/script/webhooks/deduplicate.ts |
Updated to handle both compact and legacy param formats with helper functions |
languageservice/package.json |
Updated scripts to include new strings.json in minification and point to new update-webhooks script |
docs/json-data-files.md |
Comprehensive documentation of compact format, string interning, and optimization details |
.gitignore |
Updated patterns for new full.json and validation-complete marker files |
.github/workflows/buildtest.yml |
Added separate validate-webhooks job to verify optimization correctness |
languageservice/script/webhooks/index.ts |
Removed (replaced by update-webhooks.ts) |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
d953314 to
c034e38
Compare
- Convert params to compact array format (type-based dispatch) - Intern duplicate property names into string table - Use negative indices for object references to distinguish from string indices - Rename objects.json → webhooks.objects.json, add webhooks.strings.json - Move event filters to JSON for maintainability - Add CI validation job to verify optimization correctness Reduces combined minified size by ~67% (453 KB → 148 KB), ~27% gzipped (23 KB → 17 KB).
c034e38 to
c04c1b2
Compare
Overview
This PR optimizes the webhook payload JSON files used for GitHub Actions autocompletion and hover documentation.
Total JSON payload (gzipped): 42 KB → 36 KB (15% reduction)
Changes
New Optimizations
Compact array format — Params converted from verbose objects to type-dispatched arrays:
String interning — Duplicate property names stored in
webhooks.strings.jsonand referenced by non-negative indexNegative indices for objects — Object references now use negative numbers (
-(index + 1)) to distinguish from string indices, allowing all duplicate names to be internedFile Renames
objects.json→webhooks.objects.jsonwebhooks.strings.jsonfor interned property namesOther Changes
event-filters.jsonfor maintainability"//"comment key towebhooks.jsonpointing to documentationvalidate-webhooks) to verify optimization correctnessSize Reduction
Testing
webhooks.full.json) to ensure no data lossDocumentation
Updated
docs/json-data-files.mdwith: