fix: resolve iOS E2E test failures in block inserter#338
Merged
Conversation
Blocks like Paragraph appear in multiple inserter sections (most-used and their category section), causing `app.buttons["Paragraph"]` to match multiple elements and fail with "Multiple matching elements found". Use `.firstMatch` since both buttons perform the same insert action.
dcalhoun
added
the
[Type] Automated Testing
The test-ios-e2e target only copied dist/ into the iOS bundle directory when ios/Sources/GutenbergKit/Gutenberg/ didn't exist. Since that directory is checked into git, it always exists on CI checkout, so the freshly built dist/ from the build-react pipeline step was never copied over the stale git-tracked bundle. Always replace the iOS bundle with the current dist/ contents to ensure E2E tests run against the latest JS build.
dcalhoun
changed the title
dcalhoun
added a commit
that referenced
this pull request
Feb 26, 2026
The Android assets directory is checked into git, so the conditional copy (only when the directory doesn't exist) would always skip, leaving stale build output. Always clean and replace with the current dist/ contents, matching the fix applied to iOS in PR #338.
dcalhoun
added a commit
that referenced
this pull request
Feb 27, 2026
* test: Add Espresso Web and Compose UI Test dependencies for Android E2E
Add espresso-web to the version catalog and wire up the test
dependencies needed by the instrumented E2E suite: espresso-web for
WebView DOM interaction, compose-ui-test-junit4 for Compose semantics
assertions, and ui-test-manifest for the debug variant. Disable
animations in testOptions to avoid flakiness.
* test: Add Android E2E tests for editor loading and undo/redo
Add EditorTestHelpers (mirrors iOS EditorUITestHelpers) and
EditorInteractionTest (mirrors iOS EditorInteractionUITest) with two
test cases:
- testEditorWebViewBecomesVisible: navigates Main → SitePreparation →
Editor and verifies the Gutenberg WebView loads.
- testUndoRedoAfterTyping: full native-to-web bridge round-trip —
types in title, inserts a Paragraph block via the web inserter,
types in content, then verifies Undo/Redo toolbar button state
propagates correctly through the bridge.
Key implementation details:
- Uses document.execCommand('insertText') instead of Espresso Web's
webKeys(), which fails on Gutenberg's contenteditable rich text.
- Clicks WebView toolbar buttons via JS pointer/mouse event dispatch
for React synthetic event compatibility.
- Inserts blocks through the web block inserter (Add block → Paragraph)
since Android has no native block inserter.
Removes the auto-generated ExampleInstrumentedTest placeholder.
* test: Add Makefile targets for Android E2E tests
Add test-android-e2e (production build) and test-android-e2e-dev
(Vite dev server) targets, mirroring the existing test-ios-e2e /
test-ios-e2e-dev pattern.
Includes an ENSURE_ANDROID_DEVICE macro that checks for any
connected device via adb and auto-boots the first available AVD
if none is found.
* fix: Always replace Android assets before E2E tests
The Android assets directory is checked into git, so the conditional
copy (only when the directory doesn't exist) would always skip,
leaving stale build output. Always clean and replace with the
current dist/ contents, matching the fix applied to iOS in PR #338.
* fix: Restore emulator animations after E2E tests
Replace `testOptions { animationsDisabled = true }` with a custom
DisableAnimationsRule that saves the original animation scale values,
disables them for the test, and restores them in a finally block.
The Gradle setting permanently disabled animations on the emulator
with no cleanup. The rule-based approach ensures animations are
always restored, even if tests fail.
* fix: Replace class-based selectors with aria attributes in Android E2E tests
Use semantic selectors (aria-label, aria-modal, role, placeholder) instead
of CSS class names so tests are resilient to styling refactors.
- Scope ADD_BLOCK_SELECTOR to [aria-label='Editor toolbar'] to
disambiguate the toolbar inserter from the inline block appender
- Replace .block-editor-inserter__popover with [role='dialog'][aria-modal]
and match block options by textContent within [role='option'] elements
- Replace textarea.editor-post-text-editor with placeholder attribute
* fix: Replace Kotlin assert() with JUnit Assert methods in Android E2E tests
Kotlin's assert() compiles to Java assert statements, which can be
silently disabled at the JVM level. On Android, assertions are disabled
by default unless explicitly enabled with -ea. Using JUnit's
Assert.assertEquals, Assert.assertTrue, and Assert.assertFalse ensures
assertions always execute during test runs.
* fix: Quote $ANDROID_HOME paths in ENSURE_ANDROID_DEVICE macro
Add quotes around $ANDROID_HOME/emulator/emulator paths so the macro
handles SDK installations in directories containing spaces.
* refactor: Move Compose test dependencies to version catalog
Move the hardcoded androidx.compose.ui:ui-test-junit4 and
ui-test-manifest dependency strings in build.gradle.kts to named
entries in libs.versions.toml for consistency with the other Compose
library declarations.
* refactor: Extract waitUntil/runCatching pattern into extension functions
Add waitUntilAsserts and waitForNodeWithText extension functions on
AndroidComposeTestRule to replace the repeated waitUntil { runCatching
{ ... }.getOrDefault(false) } pattern that appeared in five places.
* refactor: Simplify clickViaJs to use el.click() instead of full event sequence
The previous implementation dispatched five synthetic events
(pointerdown, mousedown, pointerup, mouseup, click) to ensure React
picked up the interaction. In practice, el.click() dispatches a real
click event that bubbles through the DOM, which is sufficient for
React 18's event delegation. This simplifies the JS and removes the
unnecessary PointerEvent/MouseEvent ceremony.
* refactor: Consolidate WebView element waiting into single JS-based approach
Replace the dual waiting mechanism (Espresso Web findElement + JS
querySelector) with a single JS-based approach. The Espresso Web
variant used findElement with a getText/webMatches assertion, which
can fail on elements that don't pass Espresso's built-in visibility
check. The JS approach using document.querySelector is simpler and
handles both visible and non-visible elements consistently.
This removes the webMatches, getText, and notNullValue imports that
were only needed by the Espresso-based variant.
* refactor: Use Espresso Web XPath locator for block insertion
Replace the hand-rolled JS that iterates over role="option" elements
to find and click a block by name with Espresso Web's findElement using
an XPath locator. This is more idiomatic and leverages the framework's
built-in element location instead of custom JS string matching.
The dialog wait still uses the JS-based waitForWebViewElement to ensure
the inserter is rendered before attempting the XPath lookup.
* refactor: Extract inline inserter dialog selectors into named values
The insertBlock method had two inline selectors — a CSS selector for
the inserter dialog and an XPath for matching block options by name.
Extract the CSS selector to INSERTER_DIALOG_SELECTOR (alongside the
other named constants) and the XPath template to an
inserterOptionXpath helper function for clarity.
* refactor: Extract runJs() helper to deduplicate WebView JS execution
Centralizes the repeated `onWebView().forceJavascriptEnabled()
.perform(script(js)).get()` boilerplate into a single `runJs()`
method, used by clickViaJs, typeViaExecCommand, readTextViaJs,
and waitForConditionViaJs.
* fix: Narrow catch (Throwable) to catch (Exception) in waitForConditionViaJs
Avoids silently swallowing JVM errors like OutOfMemoryError or
StackOverflowError during WebView polling.
* refactor: Use Kotlin multiline strings for JS snippets
Replace string concatenation with trimIndent() multiline strings
in clickViaJs, typeViaExecCommand, readTextViaJs, and
waitForWebViewElement for improved readability.
* fix: Add boot timeout to ENSURE_ANDROID_DEVICE macro
Prevents the emulator boot loop from hanging indefinitely by
failing after 120 seconds (60 iterations x 2s sleep).
* fix: Default to 1.0 for missing animation settings in DisableAnimationsRule
When a device has no value for an animation scale setting, adb
returns the string "null". Restoring "null" as the value could
leave animations in an unexpected state. Default to "1.0" instead.
* fix: Check execCommand return value in typeViaExecCommand
Verify that document.execCommand('insertText') returns true and
throw an AssertionError if it fails, catching silent text insertion
failures.
* refactor: Simplify readTextViaJs to only handle textarea elements
All callers target Code Editor textarea fields, so the
contenteditable branch (innerText/textContent) is unused.
Simplify to just return el.value.
* fix: Kill emulator process on boot timeout in ENSURE_ANDROID_DEVICE
Store the background emulator PID and kill it if the boot times out,
preventing an orphaned emulator process from lingering after failure.
* refactor: Add EditorTestRule type alias for verbose compose rule type
Introduces a top-level type alias to reduce repetition of the
AndroidComposeTestRule<ActivityScenarioRule<MainActivity>, MainActivity>
type throughout EditorTestHelpers.
* refactor: Replace JS workarounds with standard Espresso Web APIs
Remove clickViaJs and JS-based waitForWebViewElement in favor of
standard Espresso Web findElement/webClick calls. Testing confirmed
that Espresso Web's visibility checks pass for Gutenberg toolbar
elements, making the JS bypasses unnecessary.
The execCommand('insertText') workaround for text input remains, as
Espresso Web's webKeys() genuinely fails on Gutenberg's contenteditable
with "Cannot set the selection end".
kean
approved these changes
Feb 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What?
Fix iOS E2E test failures (
testUndoRedoAfterTyping,testInsertImageBlock) when interacting with the block inserter.Why?
Two separate issues caused the failures:
CI: stale JS bundle — The
test-ios-e2eMakefile target only copieddist/into the iOS bundle directory (ios/Sources/GutenbergKit/Gutenberg/) when that directory didn't exist. Since it's checked into git, it always exists on CI checkout, so the freshly builtdist/from thebuild-reactpipeline step was never copied over the stale git-tracked bundle. This caused blocks to not appear in the inserter at all on CI.Duplicate button matches — Blocks like "Paragraph" intentionally appear in multiple inserter sections (both
gbk-most-usedand their category section). When the test helper queriesapp.buttons["Paragraph"], XCTest finds both instances and fails with "Multiple matching elements found".How?
dist/contents before running E2E tests, instead of conditionally skipping the copy when the directory already exists..firstMatchon the button query inEditorUITestHelpers.insertBlock, since both matching buttons insert the same block.Testing Instructions
make test-ios-e2eAccessibility Testing Instructions
No accessibility-facing UI changes.