Skip to content

feat(sync): add delayed task submission#1506

Open
jpnurmi wants to merge 21 commits intomasterfrom
jpnurmi/feat/delayed-task
Open

feat(sync): add delayed task submission#1506
jpnurmi wants to merge 21 commits intomasterfrom
jpnurmi/feat/delayed-task

Conversation

@jpnurmi
Copy link
Collaborator

@jpnurmi jpnurmi commented Feb 6, 2026

HTTP retries (#1520) need to throttle on startup (100ms) and then back off exponentially (15min, 30min, 1h, ...). This requires the background worker to support deferred task execution.

Add sentry__bgworker_submit_delayed() to defer task execution by a given number of milliseconds. The task queue becomes a sorted linked list by deadline, with a fast path for the common FIFO case (immediate tasks). This also required changes to flush, shutdown, and foreach-matching to handle delayed tasks as appropriate.

// delay 100ms
sentry__bgworker_submit_delayed(bgworker, http_retry_task, sentry_envelope_free, envelope, 100);

Split from #1520 because the background worker changes are non-trivial and easier to review in isolation - for both humans and AI review bots.

#skip-changelog (internal)

@github-actions
Copy link

github-actions bot commented Feb 6, 2026

Messages
📖 Do not forget to update Sentry-docs with your feature once the pull request gets approved.

Generated by 🚫 dangerJS against 7a15acc

@jpnurmi
Copy link
Collaborator Author

jpnurmi commented Feb 6, 2026

@sentry review

@jpnurmi
Copy link
Collaborator Author

jpnurmi commented Feb 6, 2026

@cursor review

Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

jpnurmi and others added 5 commits February 13, 2026 18:32
Add sentry__bgworker_submit_delayed() to defer task execution by a
given number of milliseconds. Tasks are sorted by readiness time using
monotonic timestamps, so a ready delayed task is not bypassed by a
later-submitted immediate task. On shutdown, tasks that exceed the
deadline (started + timeout) are pruned while the rest execute normally.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The flush marker now uses the last task's deadline (capped at the flush
timeout) so it sorts after all current tasks including delayed ones.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Extract the core submission logic into submit_at which takes an absolute
execute_after time. submit and submit_delayed become thin wrappers.

Use submit_at in bgworker_delayed_tasks to pin all tasks to a single
base timestamp, making the test ordering deterministic regardless of OS
preemption between submissions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move delayed task cleanup from the shutdown pre-prune pass into the
worker thread itself. When the worker encounters a delayed task that
is not yet ready and shutdown has been signaled, it discards all
remaining tasks. This catches tasks submitted during execution that
the one-time pre-prune in shutdown could not see.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Instead of explicitly discarding delayed tasks in worker_thread,
teach sentry__bgworker_is_done to treat pending delayed tasks as
done when !running. Remaining tasks are cleaned up by decref.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
jpnurmi and others added 2 commits February 15, 2026 13:19
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When a delayed task's deadline exceeds the flush timeout, don't use it
to delay the flush sentinel. Such tasks cannot complete within the
timeout anyway, and capping to the timeout caused the sentinel to race
with the caller's deadline.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@jpnurmi
Copy link
Collaborator Author

jpnurmi commented Feb 16, 2026

@sentry review

@jpnurmi
Copy link
Collaborator Author

jpnurmi commented Feb 16, 2026

@cursor review

jpnurmi and others added 2 commits February 16, 2026 20:31
The previous fix only checked last_task, so a far-future tail caused
delay_ms=0 which skipped eligible delayed tasks earlier in the queue.
Walk from first_task to find the last task due within the timeout.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Large delay_ms values could wrap execute_after into the past, causing
immediate execution. Use saturating addition capped at UINT64_MAX.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@jpnurmi
Copy link
Collaborator Author

jpnurmi commented Feb 16, 2026

@cursor review

When foreach_matching removes current_task from the queue, submit_at
would still use it as insertion predecessor, orphaning the new task.
Clear current_task so submit_at falls back to first_task.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@jpnurmi
Copy link
Collaborator Author

jpnurmi commented Feb 17, 2026

@cursor review

bgworker_shutdown used sentry__bgworker_is_done() which includes a
time check: sentry__monotonic_time() < first_task->execute_after.
This condition can flip from true to false as wall time advances past
execute_after. The worker thread exits when is_done is momentarily
true, but by the time shutdown re-checks after the done_signal wake,
time has crossed the threshold and is_done returns false. Shutdown
then keeps polling until timeout, reporting failure even though the
worker already exited cleanly.

The full is_done check (task queue + time + running flag) is only
needed by the worker thread to decide when to stop processing.
Shutdown just needs to know whether the worker has exited, which is
exactly what the running flag tells it.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@jpnurmi
Copy link
Collaborator Author

jpnurmi commented Feb 17, 2026

@cursor review

The !running check was placed before the timeout check, so
thread_join could block indefinitely and bypass the requested
timeout. Move the timeout check first so shutdown always detaches
if the deadline has passed, and only joins within the timeout window.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@jpnurmi
Copy link
Collaborator Author

jpnurmi commented Feb 17, 2026

@cursor review

jpnurmi and others added 2 commits February 17, 2026 09:21
Exercises the fix from 1adbbd2: foreach_matching clears
current_task when it drops the executing task, so a subsequent
submit_at links from first_task instead of a stale pointer.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Exercises the fix from c24c7f8: foreach_matching updates
current_task->next_task when it drops the successor, so a
subsequent submit_at walks valid pointers during insertion.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

jpnurmi and others added 2 commits February 17, 2026 11:18
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
submit_at inserts after current_task, which can leave the head
temporarily unsorted. Start the scan from the sorted tail.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@jpnurmi
Copy link
Collaborator Author

jpnurmi commented Feb 17, 2026

@sentry review

@jpnurmi
Copy link
Collaborator Author

jpnurmi commented Feb 17, 2026

@cursor review

Large delays that exceed UINT32_MAX wrap on cast, causing unnecessary
wakeups. Clamp to UINT32_MAX so the worker sleeps the maximum duration
instead of busy-looping.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@jpnurmi
Copy link
Collaborator Author

jpnurmi commented Feb 17, 2026

@cursor review

Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

@jpnurmi jpnurmi marked this pull request as ready for review February 17, 2026 11:22
@jpnurmi jpnurmi changed the title feat(sync): add delayed task submission for throttling feat(sync): add delayed task submission Feb 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

Comments