From 267dbea15db6ba70cfc99df61305ef2a81fd256f Mon Sep 17 00:00:00 2001 From: engalar Date: Fri, 3 Apr 2026 13:38:32 +0800 Subject: [PATCH 1/2] docs: add MDL i18n design proposal MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Proposes syntax-layer i18n support for MDL: inline translation map literals `{ en_US: 'Hello', zh_CN: '你好' }`, DESCRIBE WITH TRANSLATIONS output mode, and SHOW TRANSLATIONS query command. Co-Authored-By: Claude Opus 4.6 (1M context) --- docs/plans/2026-04-03-mdl-i18n-design.md | 210 +++++++++++++++++++++++ 1 file changed, 210 insertions(+) create mode 100644 docs/plans/2026-04-03-mdl-i18n-design.md diff --git a/docs/plans/2026-04-03-mdl-i18n-design.md b/docs/plans/2026-04-03-mdl-i18n-design.md new file mode 100644 index 0000000..6ef6cfc --- /dev/null +++ b/docs/plans/2026-04-03-mdl-i18n-design.md @@ -0,0 +1,210 @@ +# MDL Internationalization (i18n) Support + +**Date:** 2026-04-03 +**Status:** Proposal +**Author:** @anthropics/claude-code + +## Problem + +MDL currently handles all translatable text fields (page titles, widget captions, enumeration captions, microflow message templates) as single-language strings. When creating or describing model elements, only the default language is read or written. All other translations are silently dropped. + +This means: +- `DESCRIBE PAGE` output loses translations — roundtripping a page strips non-default languages +- `CREATE PAGE` can only set one language — multi-language projects require Studio Pro for translation +- No way to audit translation coverage from the CLI + +Mendix stores translations as `Texts$Text` objects containing an array of `Texts$Translation` entries (one per language). The mxcli internal model (`model.Text`) already represents translations as `map[string]string`, and the BSON reader/writer already handles multi-language serialization. The gap is purely at the MDL syntax and command layer. + +## Scope + +**In scope (syntax-layer extension):** +- Inline multi-language text literal syntax for CREATE/ALTER +- DESCRIBE WITH TRANSLATIONS output mode +- SHOW TRANSLATIONS query command +- Writer changes to serialize multi-language BSON correctly + +**Out of scope:** +- Batch export/import (CSV, XLIFF) — future proposal +- ALTER TRANSLATION standalone command — future proposal +- Translation memory or machine translation integration + +## Design + +### 1. Translated Text Literal Syntax + +Any MDL property that accepts a string literal `'text'` can alternatively accept a translation map: + +```sql +-- Single language (backward compatible, unchanged) +Title: 'Hello World' + +-- Multi-language +Title: { + en_US: 'Hello World', + zh_CN: '你好世界', + nl_NL: 'Hallo Wereld' +} +``` + +**Grammar (ANTLR4):** + +```antlr +translatedText + : STRING_LITERAL + | '{' translationEntry (',' translationEntry)* ','? '}' + ; + +translationEntry + : IDENTIFIER ':' STRING_LITERAL + ; +``` + +**AST node:** + +```go +type TranslatedText struct { + Translations map[string]string // languageCode → text + IsMultiLang bool // false = single bare string +} +``` + +**Semantics:** +- Bare string `'text'` writes to the project's `DefaultLanguageCode`. Existing translations in other languages are preserved. +- Map `{ lang: 'text', ... }` writes the specified languages. Languages not mentioned in the map are preserved (merge, not replace). +- No syntax for deleting a translation (use Studio Pro). + +### 2. DESCRIBE WITH TRANSLATIONS + +```sql +-- Default: single language output (backward compatible) +DESCRIBE PAGE Module.MyPage; +-- Output: Title: 'Hello World' + +-- New: all translations +DESCRIBE PAGE Module.MyPage WITH TRANSLATIONS; +-- Output: +-- Title: { +-- en_US: 'Hello World', +-- zh_CN: '你好世界' +-- } +``` + +**Rules:** +- Without `WITH TRANSLATIONS`: outputs only the default language as a bare string (current behavior). +- With `WITH TRANSLATIONS`: if only one language exists, still uses bare string; if ≥2 languages, uses map syntax. +- Output must be re-parseable by the MDL parser (roundtrip guarantee). + +**Grammar:** + +```antlr +describeStatement + : DESCRIBE objectType qualifiedName withTranslationsClause? + ; + +withTranslationsClause + : WITH TRANSLATIONS + ; +``` + +**Affected commands:** +- DESCRIBE PAGE / SNIPPET — Title, widget Caption, Placeholder +- DESCRIBE ENTITY — validation rule messages +- DESCRIBE MICROFLOW / NANOFLOW — LogMessage, ShowMessage, ValidationFeedback templates +- DESCRIBE ENUMERATION — value captions +- DESCRIBE WORKFLOW — task names, descriptions, outcome captions + +### 3. SHOW TRANSLATIONS + +```sql +-- All translations in a module +SHOW TRANSLATIONS IN Module; + +-- Only missing translations +SHOW TRANSLATIONS IN Module MISSING; + +-- All translations project-wide +SHOW TRANSLATIONS MISSING; +``` + +**Output (tabular):** + +``` +Element Context en_US zh_CN nl_NL +───────────────────────────────────────────────────────────────────────────── +Module.MyPage page_title Hello World 你好世界 ✗ +Module.MyPage.SaveButton caption Save 保存 ✗ +Module.Status.Active enum_caption Active 活跃 ✗ +``` + +`✗` indicates a missing translation. The `MISSING` filter shows only rows with at least one gap. + +**Implementation:** Reuses the existing catalog `strings` FTS5 table. Pivots rows by language code into a wide-format table. Requires `REFRESH CATALOG FULL` to index strings first. + +### 4. Writer Layer Changes + +When executing CREATE/ALTER with multi-language text, the writer serializes all provided translations into the standard Mendix BSON format: + +```go +titleItems := bson.A{int32(2)} // marker for non-empty +for langCode, text := range translatedText.Translations { + titleItems = append(titleItems, bson.D{ + {Key: "$ID", Value: generateUUID()}, + {Key: "$Type", Value: "Texts$Translation"}, + {Key: "LanguageCode", Value: langCode}, + {Key: "Text", Value: text}, + }) +} +``` + +**Merge semantics for bare strings:** +When a bare string `'text'` is used, the writer must: +1. Read the existing `Texts$Text` from the MPR +2. Update only the `DefaultLanguageCode` entry +3. Preserve all other language entries unchanged + +**Affected writer functions:** +- `writer_pages.go` — Page Title, widget Caption/Placeholder +- `writer_enumeration.go` — EnumerationValue Caption +- `writer_microflow.go` — StringTemplate (log/show/validation messages) +- `writer_widgets.go` — all widget Caption/Placeholder properties + +## Translatable Fields Inventory + +The following fields use `Texts$Text` and are affected by this proposal: + +| Category | StringContext | Count | Examples | +|----------|-------------|-------|---------| +| Page metadata | `page_title` | 1 | Page.Title | +| Enumeration values | `enum_caption` | per value | EnumerationValue.Caption | +| Microflow actions | `log_message`, `show_message`, `validation_message` | 3 | LogMessageAction, ShowMessageAction | +| Workflow objects | `task_name`, `task_description`, `outcome_caption`, `activity_caption` | 4 | UserTask.Name, UserTask.Description | +| Widget properties | `caption`, `placeholder` | 7+ | ActionButton.Caption, TextInput.Placeholder | + +**Note:** Widget-level translations (caption, placeholder) are not currently indexed in the catalog `strings` table. A follow-up task should extend `catalog/builder_strings.go` to extract these. + +## Implementation Phases + +| Phase | Scope | Dependency | +|-------|-------|------------| +| **P1** | Grammar + AST: `translatedText` rule, `TranslatedText` node | None | +| **P2** | Visitor: parse `{ lang: 'text' }` into AST | P1 | +| **P3** | DESCRIBE WITH TRANSLATIONS: all describe commands output multi-language | P1 (reuses AST) | +| **P4** | Writer: CREATE/ALTER write multi-language BSON | P1 + P2 | +| **P5** | SHOW TRANSLATIONS: catalog query command | None (independent) | +| **P6** | Widget translation indexing: extend catalog builder for widget-level translations | P5 | + +Each phase is independently deliverable and testable. + +## Compatibility + +- **Backward compatible**: existing MDL scripts with bare strings continue to work identically. +- **Forward compatible**: MDL scripts using `{ lang: 'text' }` syntax will fail gracefully on older mxcli versions with a parse error pointing to the `{` token. +- **DESCRIBE roundtrip**: `DESCRIBE ... WITH TRANSLATIONS` output can be fed back to `CREATE OR REPLACE` to reproduce the same translations. + +## Risks + +| Risk | Mitigation | +|------|-----------| +| `{` ambiguity with widget body blocks | Grammar context: `translatedText` only appears in property value position, not statement position. Widget bodies follow `)` not `:`. | +| Translation ordering in BSON | Mendix does not depend on translation order within `Items` array. Sort by language code for deterministic output. | +| Large translation maps cluttering DESCRIBE output | `WITH TRANSLATIONS` is opt-in; default remains single-language. | From 0372b516ed2c927616d0f8d2edb9da940cf78965 Mon Sep 17 00:00:00 2001 From: engalar Date: Fri, 3 Apr 2026 14:43:12 +0800 Subject: [PATCH 2/2] docs: revise i18n design per review feedback - Drop SHOW TRANSLATIONS (overlaps with SHOW LANGUAGES + QUAL005) - Add concrete ANTLR integration into propertyValueV3 - Detail read-modify-write impact on writer architecture - Reorder phases: P1 = DESCRIBE WITH TRANSLATIONS (highest value, zero risk) - Add ALTER PAGE SET + translation map interaction - Sort translations by language code for deterministic output - Fix author attribution Co-Authored-By: Claude Opus 4.6 (1M context) --- docs/plans/2026-04-03-mdl-i18n-design.md | 114 ++++++++++++++--------- 1 file changed, 69 insertions(+), 45 deletions(-) diff --git a/docs/plans/2026-04-03-mdl-i18n-design.md b/docs/plans/2026-04-03-mdl-i18n-design.md index 6ef6cfc..8edaf5e 100644 --- a/docs/plans/2026-04-03-mdl-i18n-design.md +++ b/docs/plans/2026-04-03-mdl-i18n-design.md @@ -1,8 +1,8 @@ # MDL Internationalization (i18n) Support -**Date:** 2026-04-03 -**Status:** Proposal -**Author:** @anthropics/claude-code +**Date:** 2026-04-03 (updated 2026-04-03) +**Status:** Proposal (revised per review feedback) +**Author:** @engalar ## Problem @@ -11,16 +11,15 @@ MDL currently handles all translatable text fields (page titles, widget captions This means: - `DESCRIBE PAGE` output loses translations — roundtripping a page strips non-default languages - `CREATE PAGE` can only set one language — multi-language projects require Studio Pro for translation -- No way to audit translation coverage from the CLI +- No way to see all translations in context (note: `SHOW LANGUAGES` and `QUAL005 MissingTranslations` linter rule already provide language inventory and gap detection via the catalog `strings` table) Mendix stores translations as `Texts$Text` objects containing an array of `Texts$Translation` entries (one per language). The mxcli internal model (`model.Text`) already represents translations as `map[string]string`, and the BSON reader/writer already handles multi-language serialization. The gap is purely at the MDL syntax and command layer. ## Scope **In scope (syntax-layer extension):** -- Inline multi-language text literal syntax for CREATE/ALTER +- Inline multi-language text literal syntax for CREATE/ALTER/ALTER PAGE SET - DESCRIBE WITH TRANSLATIONS output mode -- SHOW TRANSLATIONS query command - Writer changes to serialize multi-language BSON correctly **Out of scope:** @@ -48,17 +47,35 @@ Title: { **Grammar (ANTLR4):** +New rule: + ```antlr -translatedText - : STRING_LITERAL - | '{' translationEntry (',' translationEntry)* ','? '}' +translationMap + : LBRACE translationEntry (COMMA translationEntry)* COMMA? RBRACE ; translationEntry - : IDENTIFIER ':' STRING_LITERAL + : IDENTIFIER COLON STRING_LITERAL ; ``` +Integration into `propertyValueV3` (MDLParser.g4 line ~1961): + +```antlr +propertyValueV3 + : STRING_LITERAL + | translationMap // NEW: { en_US: 'Hello', zh_CN: '你好' } + | NUMBER_LITERAL + | booleanLiteral + | qualifiedName + | IDENTIFIER + | H1 | H2 | H3 | H4 | H5 | H6 + | LBRACKET (expression (COMMA expression)*)? RBRACKET + ; +``` + +**Disambiguation from widget body `{`**: `translationMap` only appears inside `propertyValueV3`, which follows `COLON` or `EQUALS` in property definitions. Widget bodies (`widgetBodyV3`) follow `)` at statement level, never after `:`. The parser sees `Caption: {` and enters `propertyValueV3 → translationMap` — there is no ambiguity because `widgetBodyV3` is a separate production in `widgetStatementV3` that requires `(...)` before `{`. + **AST node:** ```go @@ -113,34 +130,24 @@ withTranslationsClause - DESCRIBE ENUMERATION — value captions - DESCRIBE WORKFLOW — task names, descriptions, outcome captions -### 3. SHOW TRANSLATIONS - -```sql --- All translations in a module -SHOW TRANSLATIONS IN Module; +### 3. ALTER PAGE SET with Translation Maps --- Only missing translations -SHOW TRANSLATIONS IN Module MISSING; +Translation maps work in ALTER PAGE SET, enabling in-place translation updates: --- All translations project-wide -SHOW TRANSLATIONS MISSING; +```sql +ALTER PAGE Module.MyPage + SET WIDGET saveButton Caption: { en_US: 'Save', zh_CN: '保存' }; ``` -**Output (tabular):** +This reuses the `translationMap` rule inside `propertyValueV3` — no additional grammar changes needed since ALTER PAGE SET already uses `propertyValueV3` for values. -``` -Element Context en_US zh_CN nl_NL -───────────────────────────────────────────────────────────────────────────── -Module.MyPage page_title Hello World 你好世界 ✗ -Module.MyPage.SaveButton caption Save 保存 ✗ -Module.Status.Active enum_caption Active 活跃 ✗ -``` +### 4. Relationship to Existing Translation Features -`✗` indicates a missing translation. The `MISSING` filter shows only rows with at least one gap. +`SHOW LANGUAGES` (commit a060152) already lists project languages with string counts. `QUAL005 MissingTranslations` linter rule already detects missing translations. The catalog `strings` FTS5 table already stores per-language text with `SELECT * FROM CATALOG.strings WHERE Language = 'nl_NL'`. -**Implementation:** Reuses the existing catalog `strings` FTS5 table. Pivots rows by language code into a wide-format table. Requires `REFRESH CATALOG FULL` to index strings first. +This proposal does **not** duplicate those features. It addresses the gap they cannot fill: **writing and round-tripping multi-language text in MDL syntax**. -### 4. Writer Layer Changes +### 5. Writer Layer Changes When executing CREATE/ALTER with multi-language text, the writer serializes all provided translations into the standard Mendix BSON format: @@ -156,17 +163,33 @@ for langCode, text := range translatedText.Translations { } ``` -**Merge semantics for bare strings:** -When a bare string `'text'` is used, the writer must: -1. Read the existing `Texts$Text` from the MPR -2. Update only the `DefaultLanguageCode` entry -3. Preserve all other language entries unchanged +**Merge semantics for bare strings (architectural change):** + +Currently, all writer functions construct `Texts$Text` from scratch — e.g. `writer_pages.go:219-247` builds a new `Items` array every time. Bare-string merge semantics require a **read-modify-write cycle**: + +1. Read the existing `Texts$Text` BSON from the MPR via `GetRawUnit` +2. Parse existing `Items` array to find the entry for `DefaultLanguageCode` +3. Update that entry's `Text` field (or insert if missing) +4. Preserve all other `Texts$Translation` entries unchanged +5. Write back the modified `Items` array + +This is a significant change to writer architecture. A shared helper should be introduced: -**Affected writer functions:** +```go +// mergeTranslation reads existing Texts$Text, merges new translations, returns updated BSON. +// For bare strings: translations = {defaultLang: text} +// For maps: translations = the full map +func mergeTranslation(existingBSON bson.D, translations map[string]string) bson.D +``` + +**Affected writer functions (11+ call sites):** - `writer_pages.go` — Page Title, widget Caption/Placeholder - `writer_enumeration.go` — EnumerationValue Caption - `writer_microflow.go` — StringTemplate (log/show/validation messages) - `writer_widgets.go` — all widget Caption/Placeholder properties +- `writer_widgets_action.go`, `writer_widgets_display.go`, `writer_widgets_input.go` + +**Serialization ordering:** Translations within `Items` array must be sorted by language code for deterministic BSON output and diff-friendly DESCRIBE. ## Translatable Fields Inventory @@ -184,16 +207,17 @@ The following fields use `Texts$Text` and are affected by this proposal: ## Implementation Phases -| Phase | Scope | Dependency | -|-------|-------|------------| -| **P1** | Grammar + AST: `translatedText` rule, `TranslatedText` node | None | -| **P2** | Visitor: parse `{ lang: 'text' }` into AST | P1 | -| **P3** | DESCRIBE WITH TRANSLATIONS: all describe commands output multi-language | P1 (reuses AST) | -| **P4** | Writer: CREATE/ALTER write multi-language BSON | P1 + P2 | -| **P5** | SHOW TRANSLATIONS: catalog query command | None (independent) | -| **P6** | Widget translation indexing: extend catalog builder for widget-level translations | P5 | +| Phase | Scope | Dependency | Risk | +|-------|-------|------------|------| +| **P1** | DESCRIBE WITH TRANSLATIONS: all describe commands output multi-language | None — read-only, no grammar change | Low | +| **P2** | Grammar + AST: `translationMap` rule, `TranslatedText` node | None | Low | +| **P3** | Visitor: parse `{ lang: 'text' }` into AST | P2 | Low | +| **P4** | Writer `mergeTranslation` helper + multi-lang BSON write | P3 | **High** — architectural change to writer, must test against Studio Pro | +| **P5** | Widget translation indexing: extend catalog builder for widget-level translations | None (independent) | Low | + +P1 first — highest user value, zero risk. P4 is the riskiest phase. -Each phase is independently deliverable and testable. +**Dropped**: SHOW TRANSLATIONS command — `SHOW LANGUAGES` + `QUAL005` + `SELECT ... FROM CATALOG.strings` already cover translation auditing. ## Compatibility