Skip to content

fix(Gemini): Add Usage to StepFinishEvent#1000

Open
Plytas wants to merge 2 commits intoprism-php:mainfrom
Plytas:fix/stream-step-usage
Open

fix(Gemini): Add Usage to StepFinishEvent#1000
Plytas wants to merge 2 commits intoprism-php:mainfrom
Plytas:fix/stream-step-usage

Conversation

@Plytas
Copy link
Copy Markdown

@Plytas Plytas commented Apr 2, 2026

Description

When using ->withTools()->asStream() in Gemini, we're missing token usage on the intermediate LLM call:

  User Message
       │
       ▼
  ┌─────────┐    usage: prompt=100, comp=200, thought=50
  │ LLM #1  │──→ StepFinishEvent  ❌ no usage field
  └────┬────┘
       │ tool calls
       ▼
  ┌──────────┐
  │ Tool Exe │
  └────┬─────┘
       │ tool results
       ▼
  ┌─────────┐    usage: prompt=200, comp=400, thought=0
  │ LLM #2  │──→ StepFinishEvent  ❌ no usage field
  └────┬────┘
       │ final text
       ▼
  StreamEndEvent
    usage: prompt=200, comp=400  ← ❌ only LLM #2

There is a code block in Gemini Stream handler that supposedly updates usage to include previous usage:

if ($previousUsage instanceof Usage && $this->state->usage() instanceof Usage) {
$this->state->withUsage(new Usage(
promptTokens: $previousUsage->promptTokens + $this->state->usage()->promptTokens,
completionTokens: $previousUsage->completionTokens + $this->state->usage()->completionTokens,
cacheWriteInputTokens: ($previousUsage->cacheWriteInputTokens ?? 0) + ($this->state->usage()->cacheWriteInputTokens ?? 0),
cacheReadInputTokens: ($previousUsage->cacheReadInputTokens ?? 0) + ($this->state->usage()->cacheReadInputTokens ?? 0),
thoughtTokens: ($previousUsage->thoughtTokens ?? 0) + ($this->state->usage()->thoughtTokens ?? 0)
));
}

However, this is effectively dead code as StreamEndEvent is fired before this and there's no way to retrieve this updated usage. There are no tests that hit this block.

Proposed solution

I propose to expose Usage via StepFinishEvent. This PR only adds it to Gemini provider (as that's what I have access to), but I'd be able to add it to all instances of StepFinishEvent if needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant