Skip to content

fix: AI assistant not showing logs when run selected mid-session#4475

Open
lmac-1 wants to merge 5 commits intomainfrom
fix/4380-ai-assistant-logs-not-sent
Open

fix: AI assistant not showing logs when run selected mid-session#4475
lmac-1 wants to merge 5 commits intomainfrom
fix/4380-ai-assistant-logs-not-sent

Conversation

@lmac-1
Copy link
Collaborator

@lmac-1 lmac-1 commented Feb 27, 2026

Description

Users reported that logs weren't appearing in the AI assistant even when the "Send logs" checkbox was checked. Sentry showed context.log = null errors.

The bug occurred when:

  1. User opens AI assistant for a job (no run selected)
  2. User later selects a run from history
  3. User checks "Send logs" and sends a message
  4. Logs don't appear (context.log is null)

Root cause: follow_run_id was only stored during session creation. When users selected a run mid-session, the frontend sent follow_run_id in params, but the backend wasn't updating session.meta with the new run context, so logs couldn't be fetched.

Closes #4380

Solution

Store and propagate follow_run_id during message processing:

  • Added maybe_put_follow_run_id_in_meta/2 helper in AiAssistantChannel to store follow_run_id from params in message.meta
  • Updated MessageProcessor.update_session_with_job_context/2 to propagate follow_run_id from message.meta to in-memory session.meta
  • This ensures maybe_add_run_logs can find the run_id and fetch logs during enrichment

Regression tests:

  • Channel test: Verifies follow_run_id is stored in message.meta
  • Message processor test: Verifies successful processing with follow_run_id
  • Integration test: Verifies logs are fetched when follow_run_id is added mid-session

Validation steps

  1. Open a workflow
  2. Run it
  3. But make sure no run is selected (viewed) in the canvas
  4. Select a job node
  5. Open the IDE
  6. Now open the AI Assistant
  7. Send a message to create a brand new AI session
  8. Now select a run (click the History button in the IDE and pick a run, or just run the workflow again)
  9. Now check the 'Send logs' checkbox in the AI Assistant (don't create a new AI session, continue with the session you created at step 7)
  10. Now ask the AI Assistant if it 'sees' your logs. It should now see your logs. Ask questions about the logs or ask it to give more information to make sure it's not inventing the logs.

Additional notes for the reviewer

n/a

AI Usage

Please disclose whether you've used AI anywhere in this PR (it's cool, we just
want to know!):

  • I have used Claude Code
  • I have used another model
  • I have not used AI

You can read more details in our
Responsible AI Policy

Pre-submission checklist

  • I have performed an AI review of my code (we recommend using /review
    with Claude Code)
  • I have implemented and tested all related authorization policies.
    (e.g., :owner, :admin, :editor, :viewer)
  • I have updated the changelog.
  • I have ticked a box in "AI usage" in this PR

elias-ba and others added 3 commits February 26, 2026 20:05
follow_run_id was only stored in session meta during channel JOIN.
When a user created a session without a run selected and later picked
one, the backend silently dropped follow_run_id from new_message params,
so maybe_add_run_logs never found it and context.log was null.

Now follow_run_id is stored per-message on message meta (same pattern as
unsaved_job), and the Oban worker propagates it to the in-memory session
before enrichment.
Add tests to prevent regression of the bug where logs weren't sent
to Apollo when users selected a run mid-session. Tests verify that
follow_run_id is properly stored in message.meta and propagated
during message processing for log enrichment.
@josephjclark
Copy link
Collaborator

This all seems very plausible @lmac-1 but I am still unable to reproduce this against main locally. I consistently see the logs being uploaded. I've tried following your steps several times - I must be doing something subtly wrong? I presume you've managed to repro?

I for one would be happy to merge it and see. We'd have to track it closely in prod.

@lmac-1
Copy link
Collaborator Author

lmac-1 commented Mar 2, 2026

Loom here to see it in action before and after the fix. The steps are a bit fiddly. https://www.loom.com/share/a569fbb2e34f4fb3bb006aea028ec50d

@josephjclark
Copy link
Collaborator

Ah fantastic @lmac-1 thank you - I'll take a look in the morning

@josephjclark
Copy link
Collaborator

That's amazing @lmac-1 I can repro!!!

Copy link
Collaborator

@josephjclark josephjclark left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cannot comment on the technical solution but I've tested this and verified

What makes this hard is a litany of bugs around logging (from being able to tick Send Logs when none are available in the first place, to the assistant happily lying about the logs it receives). And the use-case reproduced feels a bit niche - is this the problem that's being reporting in production?

But @lmac-1 has 100% identified a bug and 100% fixed it as far as I'm concerned, so it's a Yes from me, Jim.

Copy link
Collaborator

@midigofrank midigofrank left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nicely done

@midigofrank
Copy link
Collaborator

@lmac-1 looks like you have one failing test.

lmac-1 added 2 commits March 4, 2026 15:43
The regression test added in 5a22a97 had three setup issues:
- Missing workflow and project in test context
- Missing required dataclip for run creation
- Message lookup found first user message instead of the specific
  message with follow_run_id

Fix by using workflow/project from context, creating a dataclip,
and finding the message by content to get the correct one.
@codecov
Copy link

codecov bot commented Mar 4, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.42%. Comparing base (675fda7) to head (f9b7d5a).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #4475   +/-   ##
=======================================
  Coverage   89.41%   89.42%           
=======================================
  Files         425      425           
  Lines       20245    20253    +8     
=======================================
+ Hits        18102    18111    +9     
+ Misses       2143     2142    -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@lmac-1
Copy link
Collaborator Author

lmac-1 commented Mar 4, 2026

@midigofrank tests are now fixed. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: New Issues

Development

Successfully merging this pull request may close these issues.

AI assistant not sending logs in prod?

4 participants