fix: AI assistant not showing logs when run selected mid-session#4475
fix: AI assistant not showing logs when run selected mid-session#4475
Conversation
follow_run_id was only stored in session meta during channel JOIN. When a user created a session without a run selected and later picked one, the backend silently dropped follow_run_id from new_message params, so maybe_add_run_logs never found it and context.log was null. Now follow_run_id is stored per-message on message meta (same pattern as unsaved_job), and the Oban worker propagates it to the in-memory session before enrichment.
Add tests to prevent regression of the bug where logs weren't sent to Apollo when users selected a run mid-session. Tests verify that follow_run_id is properly stored in message.meta and propagated during message processing for log enrichment.
|
This all seems very plausible @lmac-1 but I am still unable to reproduce this against main locally. I consistently see the logs being uploaded. I've tried following your steps several times - I must be doing something subtly wrong? I presume you've managed to repro? I for one would be happy to merge it and see. We'd have to track it closely in prod. |
|
Loom here to see it in action before and after the fix. The steps are a bit fiddly. https://www.loom.com/share/a569fbb2e34f4fb3bb006aea028ec50d |
|
Ah fantastic @lmac-1 thank you - I'll take a look in the morning |
|
That's amazing @lmac-1 I can repro!!! |
josephjclark
left a comment
There was a problem hiding this comment.
Cannot comment on the technical solution but I've tested this and verified
What makes this hard is a litany of bugs around logging (from being able to tick Send Logs when none are available in the first place, to the assistant happily lying about the logs it receives). And the use-case reproduced feels a bit niche - is this the problem that's being reporting in production?
But @lmac-1 has 100% identified a bug and 100% fixed it as far as I'm concerned, so it's a Yes from me, Jim.
|
@lmac-1 looks like you have one failing test. |
The regression test added in 5a22a97 had three setup issues: - Missing workflow and project in test context - Missing required dataclip for run creation - Message lookup found first user message instead of the specific message with follow_run_id Fix by using workflow/project from context, creating a dataclip, and finding the message by content to get the correct one.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #4475 +/- ##
=======================================
Coverage 89.41% 89.42%
=======================================
Files 425 425
Lines 20245 20253 +8
=======================================
+ Hits 18102 18111 +9
+ Misses 2143 2142 -1 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
@midigofrank tests are now fixed. Thanks! |
Description
Users reported that logs weren't appearing in the AI assistant even when the "Send logs" checkbox was checked. Sentry showed
context.log = nullerrors.The bug occurred when:
Root cause:
follow_run_idwas only stored during session creation. When users selected a run mid-session, the frontend sentfollow_run_idin params, but the backend wasn't updatingsession.metawith the new run context, so logs couldn't be fetched.Closes #4380
Solution
Store and propagate follow_run_id during message processing:
maybe_put_follow_run_id_in_meta/2helper inAiAssistantChannelto storefollow_run_idfrom params inmessage.metaMessageProcessor.update_session_with_job_context/2to propagatefollow_run_idfrommessage.metato in-memorysession.metamaybe_add_run_logscan find the run_id and fetch logs during enrichmentRegression tests:
follow_run_idis stored in message.metaValidation steps
Additional notes for the reviewer
n/a
AI Usage
Please disclose whether you've used AI anywhere in this PR (it's cool, we just
want to know!):
You can read more details in our
Responsible AI Policy
Pre-submission checklist
/reviewwith Claude Code)
(e.g.,
:owner,:admin,:editor,:viewer)