refactor engineClient for Unified xlayer binary : implement transport-agnostic engine-client

[defistar]

Context

Part of the xlayer-node initiative: running kona-node (CL/derivation) and op-reth (EL) in the same OS process, replacing the HTTP Engine API hop with direct Rust channel communication.

Current architecture:

kona-node  --[HTTP Engine API]--> op-reth

Target architecture:

kona-node  --[Rust channel]--> op-reth  (same process)

This PR lays the groundwork — it does not yet implement the in-process client. It makes the existing EngineClient trait and RollupNode service ready to accept one.

---

Problem

EngineClient was defined as:

pub trait EngineClient: OpEngineApi<Optimism, Http<HyperAuthClient>> + Send + Sync { ... }

The OpEngineApi<Optimism, Http<HyperAuthClient>> supertrait hardcodes the HTTP transport into the interface. Any struct without HyperAuthClient — including an in-process channel bridge — cannot implement EngineClient. There is also no way to inject a pre-built client into RollupNode::start(); the node always constructs its own OpEngineClient internally.

---

Changes

Commit 1 — refactor(kona-engine): make EngineClient transport-agnostic

Files: client.rs, test_utils/engine_client.rs

• Removed OpEngineApi<Optimism, Http<HyperAuthClient>> supertrait from EngineClient.
• The 8 Engine API methods the task queue uses (new_payload_v2/v3/v4, fork_choice_updated_v2/v3, get_payload_v2/v3/v4) are now declared explicitly on EngineClient itself.
• impl EngineClient for OpEngineClient delegates each method to the existing HTTP providers — zero behaviour change on the HTTP path.
• impl OpEngineApi for OpEngineClient is kept intact; other callers using it directly are unaffected.
• MockEngineClient consolidated from two impl blocks into one; dead fields removed (get_payload_bodies, exchange_capabilities, etc. — unused by EngineClient).
• New test test_engine_client_is_transport_agnostic: compile-time proof that a struct with no HTTP types satisfies EngineClient and can be used as dyn EngineClient.

Commit 2 — feat(kona-node): expose injection point for pluggable engine client

Files: rpc_request_processor.rs, service/node.rs

• EngineRpcProcessor.rollup_boost_server: Arc<RollupBoostServer> → Option<Arc<RollupBoostServer>>.
  • None when using an in-process client (no RollupBoost sidecar in that topology).
  • Admin queries with None server: logged + ignored.
  • Health queries with None server: respond ServiceUnavailable (honest — the service is not present).
• create_engine_actor made generic over E: EngineClient. Accepts a pre-built Arc<E> and Option<Arc<RollupBoostServer>>. No longer fallible (client building moved out).
• RollupNode::start() unchanged in behaviour: builds OpEngineClient, extracts rollup_boost, delegates to shared body.
• New RollupNode::start_with_client<E: EngineClient + 'static>(engine_client: Arc<E>): public entry point for in-process integration. Passes None for RollupBoost.
• Private start_with_engine<E>: shared actor-wiring body. Both start() and start_with_client() converge here — no duplication.

---

What does NOT change

• HTTP Engine API path (start()) — identical behaviour, same OpEngineClient, same RollupBoost wiring.
• OpEngineApi trait and its impl on OpEngineClient — untouched.
• All other actors (derivation, network, L1 watcher, sequencer, RPC) — untouched.
• Public API of RollupNode except the addition of start_with_client.
• EngineConfig — still builds OpEngineClient for the HTTP path.

---

Testing

• All 57 kona-engine unit tests pass including the new test_engine_client_is_transport_agnostic.
• kona-node-service compiles clean (10 pre-existing warnings, zero new).

---

Follow-up (not in this PR)

1. RethInProcessClient (in okx/reth) — implements EngineClient, routes engine calls onto BeaconEngineMessage channel directly into op-reth's engine tree.
2. xlayer-node binary — starts op-reth, creates RethInProcessClient, calls rollup_node.start_with_client(Arc::new(client)).
3. Remove JWT secret / --authrpc config from unified binary (no HTTP auth needed).

[defistar]

delegates the function call to OpEngineAPI which will make http based

The delegation always existed — before it was implicit (compiler wired it via supertrait), now it's explicit (we wrote it by hand). Same machine code, different source structure.

[Vui-Chee]

Extensibility concern: trait will need modification for every new fork

The transport-agnostic refactor is the right direction. One follow-up concern worth addressing before RethInProcessClient is written.

The problem

versions.rs already has the right version-selection enums:

```rust
EngineNewPayloadVersion { V2, V3, V4 }
EngineForkchoiceVersion { V2, V3 }
EngineGetPayloadVersion { V2, V3, V4 }
```

BuildTask and SealTask already consult these enums via from_cfg(). But then the tasks switch on them to call individual versioned methods on the trait (fork_choice_updated_v2/v3, get_payload_v2/v3/v4, new_payload_v2/v3/v4).

This means every new OP Stack fork (V5, V6, ...) requires:

1. New methods added to EngineClient trait
2. New impls on OpEngineClient
3. New impls on MockEngineClient
4. New impls on RethInProcessClient (the upcoming in-process bridge)
5. New match arms in InsertTask, SealTask, BuildTask

Steps 1–4 are pure boilerplate from a leaky abstraction. Step 5 is unavoidable and desirable (compiler-enforced exhaustiveness check is a safety net).

Suggested fix: enum-dispatched methods

Replace the 8 versioned methods with 3 stable methods. The version enum (or the payload variant itself) becomes the discriminant:

```rust
pub trait EngineClient: Send + Sync {
// OpExecutionPayload variant IS the version — no vN method needed
async fn new_payload(
&self,
envelope: OpExecutionPayloadEnvelope,
) -> TransportResult;

// Version enum replaces fork_choice_updated_v2 / _v3
async fn fork_choice_updated(
    &self,
    version: EngineForkchoiceVersion,
    state: ForkchoiceState,
    attrs: Option<OpPayloadAttributes>,
) -> TransportResult<ForkchoiceUpdated>;

// Version enum replaces get_payload_v2 / _v3 / _v4; returns unified envelope
async fn get_payload(
    &self,
    version: EngineGetPayloadVersion,
    payload_id: PayloadId,
) -> TransportResult<OpExecutionPayloadEnvelope>;

// all non-versioned methods unchanged

}
```

Impact on call sites

InsertTask::execute() — the 30-line match that calls new_payload_vN collapses to:
```rust
let response = self.client.new_payload(self.envelope.clone()).await;
```
The OpExecutionPayload variant match (for constructing the OpBlock) stays — that's the compiler safety net we want.

SealTask::seal_payload() — the 30-line match that calls get_payload_vN and normalises three different return types into OpExecutionPayloadEnvelope collapses to:
```rust
let version = EngineGetPayloadVersion::from_cfg(cfg, payload_timestamp);
let payload_envelope = self.engine.get_payload(version, payload_id).await?;
```
The normalisation moves into OpEngineClient::get_payload() where it belongs.

BuildTask::start_build() — the match calling fork_choice_updated_v2/v3 collapses to:
```rust
let update = engine_client.fork_choice_updated(forkchoice_version, new_forkchoice, Some(...)).await?;
```

When V5 arrives

• Add V5 to the version enums in versions.rs ✓
• Add V5 match arm in OpEngineClient::new_payload() / get_payload() — one place each ✓
• Compiler forces new match arms in the tasks — the safety check we want ✓
• No trait changes. No new impl blocks on Mock or RethInProcessClient. ✓

Recommendation

This is worth doing before RethInProcessClient is written. Once that impl exists, this refactor touches one more file. The trait surface is the contract between kona-node and the in-process bridge — better to get it right before it has two implementors.

[defistar]

Extensibility concern: trait will need modification for every new fork

The transport-agnostic refactor is the right direction. One follow-up concern worth addressing before RethInProcessClient is written.

The problem

versions.rs already has the right version-selection enums:

rust EngineNewPayloadVersion { V2, V3, V4 } EngineForkchoiceVersion { V2, V3 } EngineGetPayloadVersion { V2, V3, V4 }

BuildTask and SealTask already consult these enums via from_cfg(). But then the tasks switch on them to call individual versioned methods on the trait (fork_choice_updated_v2/v3, get_payload_v2/v3/v4, new_payload_v2/v3/v4).

This means every new OP Stack fork (V5, V6, ...) requires:

1. New methods added to EngineClient trait
2. New impls on OpEngineClient
3. New impls on MockEngineClient
4. New impls on RethInProcessClient (the upcoming in-process bridge)
5. New match arms in InsertTask, SealTask, BuildTask

Steps 1–4 are pure boilerplate from a leaky abstraction. Step 5 is unavoidable and desirable (compiler-enforced exhaustiveness check is a safety net).

Suggested fix: enum-dispatched methods

Replace the 8 versioned methods with 3 stable methods. The version enum (or the payload variant itself) becomes the discriminant:

// Version enum replaces fork_choice_updated_v2 / _v3
async fn fork_choice_updated(
&self,
version: EngineForkchoiceVersion,
state: ForkchoiceState,
attrs: Option,
) -> TransportResult;

// Version enum replaces get_payload_v2 / _v3 / _v4; returns unified envelope
async fn get_payload(
&self,
version: EngineGetPayloadVersion,
payload_id: PayloadId,
) -> TransportResult;

// all non-versioned methods unchanged

} ```

### Impact on call sites
**`InsertTask::execute()`** — the 30-line match that calls `new_payload_vN` collapses to: ```rust let response = self.client.new_payload(self.envelope.clone()).await; ``` The `OpExecutionPayload` variant match (for constructing the `OpBlock`) stays — that's the compiler safety net we want.

**`SealTask::seal_payload()`** — the 30-line match that calls `get_payload_vN` and normalises three different return types into `OpExecutionPayloadEnvelope` collapses to: ```rust let version = EngineGetPayloadVersion::from_cfg(cfg, payload_timestamp); let payload_envelope = self.engine.get_payload(version, payload_id).await?; ``` The normalisation moves into `OpEngineClient::get_payload()` where it belongs.

**`BuildTask::start_build()`** — the match calling `fork_choice_updated_v2/v3` collapses to: ```rust let update = engine_client.fork_choice_updated(forkchoice_version, new_forkchoice, Some(...)).await?; ```

### When V5 arrives
* Add `V5` to the version enums in `versions.rs` ✓
* Add `V5` match arm in `OpEngineClient::new_payload()` / `get_payload()` — one place each ✓
* Compiler forces new match arms in the tasks — the safety check we want ✓
* **No trait changes. No new impl blocks on Mock or RethInProcessClient.** ✓

### Recommendation
This is worth doing before `RethInProcessClient` is written. Once that impl exists, this refactor touches one more file. The trait surface is the contract between `kona-node` and the in-process bridge — better to get it right before it has two implementors.

Extensibility concern: trait will need modification for every new fork

The transport-agnostic refactor is the right direction. One follow-up concern worth addressing before RethInProcessClient is written.

The problem

versions.rs already has the right version-selection enums:

rust EngineNewPayloadVersion { V2, V3, V4 } EngineForkchoiceVersion { V2, V3 } EngineGetPayloadVersion { V2, V3, V4 }

BuildTask and SealTask already consult these enums via from_cfg(). But then the tasks switch on them to call individual versioned methods on the trait (fork_choice_updated_v2/v3, get_payload_v2/v3/v4, new_payload_v2/v3/v4).

This means every new OP Stack fork (V5, V6, ...) requires:

1. New methods added to EngineClient trait
2. New impls on OpEngineClient
3. New impls on MockEngineClient
4. New impls on RethInProcessClient (the upcoming in-process bridge)
5. New match arms in InsertTask, SealTask, BuildTask

Steps 1–4 are pure boilerplate from a leaky abstraction. Step 5 is unavoidable and desirable (compiler-enforced exhaustiveness check is a safety net).

Suggested fix: enum-dispatched methods

Replace the 8 versioned methods with 3 stable methods. The version enum (or the payload variant itself) becomes the discriminant:

// Version enum replaces fork_choice_updated_v2 / _v3
async fn fork_choice_updated(
&self,
version: EngineForkchoiceVersion,
state: ForkchoiceState,
attrs: Option,
) -> TransportResult;

// Version enum replaces get_payload_v2 / _v3 / _v4; returns unified envelope
async fn get_payload(
&self,
version: EngineGetPayloadVersion,
payload_id: PayloadId,
) -> TransportResult;

// all non-versioned methods unchanged

} ```

### Impact on call sites
**`InsertTask::execute()`** — the 30-line match that calls `new_payload_vN` collapses to: ```rust let response = self.client.new_payload(self.envelope.clone()).await; ``` The `OpExecutionPayload` variant match (for constructing the `OpBlock`) stays — that's the compiler safety net we want.

**`SealTask::seal_payload()`** — the 30-line match that calls `get_payload_vN` and normalises three different return types into `OpExecutionPayloadEnvelope` collapses to: ```rust let version = EngineGetPayloadVersion::from_cfg(cfg, payload_timestamp); let payload_envelope = self.engine.get_payload(version, payload_id).await?; ``` The normalisation moves into `OpEngineClient::get_payload()` where it belongs.

**`BuildTask::start_build()`** — the match calling `fork_choice_updated_v2/v3` collapses to: ```rust let update = engine_client.fork_choice_updated(forkchoice_version, new_forkchoice, Some(...)).await?; ```

### When V5 arrives
* Add `V5` to the version enums in `versions.rs` ✓
* Add `V5` match arm in `OpEngineClient::new_payload()` / `get_payload()` — one place each ✓
* Compiler forces new match arms in the tasks — the safety check we want ✓
* **No trait changes. No new impl blocks on Mock or RethInProcessClient.** ✓

### Recommendation
This is worth doing before `RethInProcessClient` is written. Once that impl exists, this refactor touches one more file. The trait surface is the contract between `kona-node` and the in-process bridge — better to get it right before it has two implementors.

@Vui-Chee this sounds nice. will add a new task to enhance this to support versioning. probably in to same PR
and we can conclude this PR with this pending change. Trying to mantain micro PRs as lean as possible

CC @cliff0412