Skip to content

Working on Test Harness for Remote Connections#2829

Open
sawka wants to merge 7 commits intomainfrom
sawka/conn-testing
Open

Working on Test Harness for Remote Connections#2829
sawka wants to merge 7 commits intomainfrom
sawka/conn-testing

Conversation

@sawka
Copy link
Member

@sawka sawka commented Feb 6, 2026

This pull request introduces a new test-conn command-line tool for testing SSH connection flows and user input handling, along with improvements to user input provider extensibility and several configuration and validation enhancements.

  • New test-conn CLI Tool
  • User Input Provider Abstraction
  • Refactored user input handling to use a pluggable UserInputProvider interface, with a default FrontendProvider and the ability to set a custom provider
  • AI Configuration and Verbosity updates
  • Enforced that a SwapToken must be present in CommandOptsType when starting a remote shell with wsh, improving validation and error handling. (pkg/shellexec/shellexec.go)
  • Improved config directory watcher logic to log the config directory path and avoid logging errors for non-existent subdirectories

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Feb 6, 2026

Walkthrough

Adds a CLI test harness under cmd/test-conn/ with a new main (flags, commands), a CLI-based user input provider (CLIProvider), and test utilities for environment setup and connection/shell tests. Introduces a pluggable user input provider API in pkg/userinput with FrontendProvider and SetUserInputProvider and delegates top-level GetUserInput to the configured provider. Adds verbosity constants in pkg/aiusechat/uctypes and updates default verbosity use. Updates default wave AI configs with an ai:verbosity key. Adds nil checks for SwapToken in shellexec, relaxes filewatcher error handling, and adds two telemetry branches for terminal commands.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

🚥 Pre-merge checks | ✅ 1 | ❌ 2
❌ Failed checks (1 warning, 1 inconclusive)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 7.69% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Title check ❓ Inconclusive The title is vague and generic, using a broad descriptor 'Working on Test Harness' that doesn't clearly convey the specific changes in the changeset. Use a more specific title that highlights the main change, such as 'Add test-conn CLI tool and refactor user input provider abstraction'.
✅ Passed checks (1 passed)
Check name Status Explanation
Description check ✅ Passed The description directly relates to the changeset, covering the main additions: test-conn CLI tool, user input provider abstraction, shell execution validation, and configuration improvements.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch sawka/conn-testing

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Fix all issues with AI agents
In `@pkg/shellexec/shellexec.go`:
- Around line 335-337: StartWslShellProc and StartLocalShellProc currently
dereference cmdOpts.SwapToken without a nil check; add the same guard used
earlier (check if cmdOpts.SwapToken == nil and return a descriptive error like
"SwapToken is required in CommandOptsType") before any dereference in
StartWslShellProc and StartLocalShellProc so they don't panic, and ensure any
subsequent uses (e.g., in functions/methods called within those routines) can
safely assume SwapToken is non-nil.
🧹 Nitpick comments (8)
pkg/wconfig/filewatcher.go (1)

54-59: Silencing not-found errors for optional subdirectories looks correct.

Gracefully skipping non-existent config subdirectories is a sensible change. One minor note: consider using errors.Is(err, fs.ErrNotExist) instead of os.IsNotExist(err) — it's the preferred idiom since Go 1.13 and correctly unwraps wrapped errors.

♻️ Suggested change

Add to imports:

"errors"
"io/fs"
-			if err != nil && !os.IsNotExist(err) {
+			if err != nil && !errors.Is(err, fs.ErrNotExist) {

This would also allow removing the "os" import if it's not used elsewhere in the file.

pkg/userinput/userinput.go (1)

148-149: Consider guarding against nil provider.

If SetUserInputProvider(nil) is called, subsequent GetUserInput calls will panic. A nil check would be a cheap safety net.

Suggested guard
 func SetUserInputProvider(provider UserInputProvider) {
+	if provider == nil {
+		panic("userinput: SetUserInputProvider called with nil provider")
+	}
 	defaultProvider = provider
 }
cmd/test-conn/cliprovider.go (1)

20-55: Implementation handles confirm-type prompts well; text-type input not handled.

The method always prompts with "Accept? [y/n]" regardless of request.ResponseType. If a request expects free-form text input (e.g., a password prompt), this will return "yes"/"no" instead of the actual text. Fine if the test harness only encounters confirm-type prompts, but worth noting for future extensibility.

cmd/test-conn/testutil.go (4)

49-57: Non-macOS platforms require WAVETERM_DATA_HOME to be set manually.

On Linux, the conventional XDG default would be ~/.local/share/waveterm. Returning an error here means the test harness won't work out-of-the-box on Linux without extra env setup. If this is intentional (macOS-only for now), consider documenting it in the usage text. Otherwise:

Suggested Linux default
 	dataHome := os.Getenv("WAVETERM_DATA_HOME")
 	if dataHome == "" {
 		if runtime.GOOS == "darwin" {
 			dataHome = filepath.Join(homeDir, "Library", "Application Support", "waveterm"+devSuffix)
 			os.Setenv("WAVETERM_DATA_HOME", dataHome)
+		} else if runtime.GOOS == "linux" {
+			dataHome = filepath.Join(homeDir, ".local", "share", "waveterm"+devSuffix)
+			os.Setenv("WAVETERM_DATA_HOME", dataHome)
 		} else {
 			return fmt.Errorf("WAVETERM_DATA_HOME must be set on non-macOS systems")
 		}
 	}

63-105: Config watcher is started but never stopped.

wconfig.GetWatcher().Start() at line 101 is called, but there's no corresponding cleanup (e.g., defer wconfig.GetWatcher().Stop() or similar). For a short-lived CLI tool this likely won't cause issues in practice, but it may leave goroutines/file watchers lingering until process exit.


223-236: SwapToken is missing a JWT token entry.

StartRemoteShellProc (shellexec.go line 453) reads cmdOpts.SwapToken.Env[wavebase.WaveJwtTokenVarName] to optionally inject a JWT into the shell environment. The token is created here without a JWT entry, and ForceJwt is false in cmdOpts, so the current flow is safe. However, if this test is meant to exercise the full wsh exec path including JWT-authenticated communication, you may want to generate and include a JWT token:

rpcCtx := wshrpc.RpcContext{Conn: connName}
jwtToken, _ := wshutil.MakeClientJWTToken(rpcCtx)
swapToken.Env[wavebase.WaveJwtTokenVarName] = jwtToken

175-187: Sleep-then-read is inherently racy for capturing command output.

The 500ms sleep may not be sufficient on slow connections or loaded systems, and a single Read call may not capture all output. This is acceptable for a test/debug tool, but be aware it can produce flaky results. A read loop with a short deadline would be more robust.

cmd/test-conn/main-test-conn.go (1)

26-27: Usage text: "connect" and "ssh" descriptions are potentially misleading.

Both connect and ssh map to the same testBasicConnect function, but the descriptions suggest different behavior ("with wsh" vs. plain). Since testBasicConnect always attempts wsh setup (via conn.Connect), the distinction in the help text may confuse users.

Suggested clarification
 Commands:
-  connect <user@host>            - Test basic SSH connection with wsh
-  ssh <user@host>                - Test basic SSH connection
+  connect, ssh <user@host>       - Test basic SSH connection (wsh enabled by config)

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Fix all issues with AI agents
In `@frontend/app/view/term/termwrap.ts`:
- Around line 253-262: The telemetry checks using
decodedCmd.startsWith("claude") and startsWith("opencode") can produce false
positives for commands that only share the prefix; update these checks to match
whole words only (e.g., require a trailing space or use a word-boundary regex)
before calling recordTEvent("action:term", { "action:type": "claude" }) /
recordTEvent("action:term", { "action:type": "opencode" }); specifically,
replace the bare startsWith calls on decodedCmd with either
decodedCmd.startsWith("claude ") / decodedCmd.startsWith("opencode ") or a regex
like /^\bclaude\b/ and /^\bopencode\b/ so only exact tokens trigger the
telemetry.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant