Implement graceful crawl abort via AbortController

[YusukeHirao]

Summary

Refactored the crawl abort mechanism to use the standard AbortController API instead of a simple boolean flag. This allows the dealer to stop launching new workers when abort is signaled, while allowing in-progress workers to complete naturally.

Key Changes

• Crawler abort mechanism: Replaced #aborted boolean flag with #abortController (AbortController instance)

  • Added public signal getter to expose the AbortSignal for external use
  • Updated abort() method to call abortController.abort() instead of setting a flag and immediately emitting crawlEnd
  • Removed the early-return check in the worker factory that was checking the #aborted flag

• Dealer integration: Pass the AbortSignal to the deal() function via the signal option

  • This allows the dealer to stop launching new workers when the signal is aborted
  • Currently running workers complete normally, then deal() resolves and crawlEnd is emitted

• CrawlerOrchestrator: Updated abort() method to delegate to crawler.abort() instead of archive.abort()

  • The abort signal now flows through the crawler's AbortController to the dealer

• CLI signal handlers: Updated the killed() handler to call orchestrator.abort() before garbage collection and process exit

  • This ensures the abort signal propagates through the dealer before cleaning up zombie processes

• Documentation: Updated ARCHITECTURE.md to document the new abort flow and mechanism

Implementation Details

The new abort flow is:

1. CLI signal handler (SIGINT/SIGHUP/etc) → calls CrawlerOrchestrator.abort()
2. CrawlerOrchestrator.abort() → calls Crawler.abort()
3. Crawler.abort() → calls AbortController.abort()
4. The AbortSignal is passed to deal(), which stops launching new workers
5. In-progress workers complete normally
6. deal() resolves → crawlEnd event is emitted via normal completion path

This is more robust than the previous approach because:

• It uses standard web APIs (AbortController/AbortSignal)
• The dealer can properly handle the abort signal and stop queuing work
• Workers aren't forcefully interrupted mid-operation
• The crawlEnd event is emitted through the normal completion path, not immediately on abort

Tests Added

Four new test cases verify the abort behavior:

• AbortSignal becomes aborted after abort() is called
• crawlEnd event is emitted on normal deal completion
• Multiple abort() calls don't throw errors
• signal getter returns an AbortSignal instance

https://claude.ai/code/session_015YfPBKbEv53mMTkgPxGCm5