AI-powered duplicate detection for auto-triage#470
Conversation
…iage-duplicate-cloud
…iage-duplicate-cloud
|
@eshurakov ccing you since I'm modifying some |
![kiloconnect[bot]](https://avatars.githubusercontent.com/in/2193792?s=60&v=4){kind=small}
Code Review SummaryStatus: 1 Issue Found | Recommendation: Address before merge Overview
Issue Details (click to expand)WARNING
Other Observations (not in diff)Issues found in new/unchanged code that are worth noting:
Files Reviewed (10 files)
|
Add timeout protection, low-confidence guard, and hallucination handling to the AI duplicate verification flow. - Wrap processStream in Promise.race with a 2-minute timeout to prevent the Durable Object from hanging indefinitely on stalled SSE streams; clear the timer in a finally block - Reject duplicate verdicts where AI confidence is below 0.5 to avoid false positives from uncertain model responses - Treat AI-hallucinated issue numbers (not present in the candidate set) as non-duplicates instead of propagating an unreliable result - Remove optional chaining on matchedTicket fields now that the hallucination case exits early, making the happy path explicit - Escape Markdown special characters in AI reasoning before embedding it in GitHub comments to prevent formatting corruption
| @@ -79,8 +84,30 @@ export class TriageOrchestrator extends DurableObject<Env> { | |||
| await this.ctx.storage.setAlarm(Date.now() + alarmTimeout); | |||
There was a problem hiding this comment.
[WARNING]: Alarm timeout no longer accounts for the full pipeline duration
The alarm budget comment (lines 80-82) says the 2-minute buffer covers "duplicate check + classification + label/comment API calls". However, the new verifyDuplicatesWithAI() step has its own 2-minute timeout (line 316), which means the worst-case pipeline is now:
- Embedding HTTP call (~seconds)
- AI duplicate verification (up to 2 min)
- Classification (up to 5 min default)
- Label/comment API calls (~seconds)
Total worst case ≈ 7+ minutes, but the alarm is set to classificationTimeout + 120_000 = 7 minutes exactly, leaving no buffer for the overhead. Consider adding the duplicate verification timeout to the alarm budget:
| await this.ctx.storage.setAlarm(Date.now() + alarmTimeout); | |
| await this.ctx.storage.setAlarm(Date.now() + alarmTimeout + 120_000); |
Or better, update alarmTimeout on line 83 to include the duplicate verification budget explicitly.
| @@ -94,7 +94,8 @@ export function AutoTriageConfigForm({ organizationId }: AutoTriageConfigFormPro | |||
| }, | |||
| onError: error => { | |||
There was a problem hiding this comment.
[WARNING]: Unused error parameter will cause a lint error
The error parameter was previously used (error.message) but is now unused since the description was replaced with a static string. With @typescript-eslint/no-unused-vars configured as 'error' in eslint.config.mjs, this will fail the build.
This same issue applies to all 4 onError callbacks in this file (lines 95, 110, 127, 142). Either prefix with _ or remove the parameter:
| onError: error => { | |
| onError: _error => { |
| @@ -138,7 +138,8 @@ export function AutoTriageTicketsCard({ organizationId }: AutoTriageTicketsCardP | |||
| }, | |||
| onError: error => { | |||
There was a problem hiding this comment.
[WARNING]: Unused error parameter will cause a lint error
Same issue as in AutoTriageConfigForm.tsx — the error parameter is no longer used after replacing error.message with a static string. This applies to all 4 onError callbacks in this file (lines 139, 165, 194, 220).
| onError: error => { | |
| onError: _error => { |
| // No githubRepo — the prompt is pure text, no codebase access needed. | ||
| const sessionInput = { | ||
| kilocodeOrganizationId: this.state.owner.type === 'org' ? this.state.owner.id : undefined, | ||
| prompt, | ||
| mode: 'ask' as const, | ||
| model: this.state.sessionInput.modelSlug, | ||
| createdOnPlatform: 'auto-triage', | ||
| }; | ||
|
|
||
| const cloudAgentClient = new CloudAgentClient(this.env.CLOUD_AGENT_URL, this.state.authToken); |
There was a problem hiding this comment.
If you don't need repo access, it may be faster and easier to just go directly to LLM. Search for sendProxiedChatCompletion
| Error: {ticket.error_message} | ||
| Error:{' '} | ||
| { | ||
| 'Something went wrong. Please try again or contact support if the problem persists.' |
There was a problem hiding this comment.
[WARNING]: This admin-facing error display now shows a hardcoded generic message instead of the actual ticket.error_message. While hiding raw errors in toast notifications (user-facing) makes sense, this component appears to be an admin/dashboard view where the real error message is valuable for debugging.
Consider keeping the original ticket.error_message here, or showing both (generic message + details in a collapsible/tooltip).
| 'Something went wrong. Please try again or contact support if the problem persists.' | |
| ticket.error_message |
2 participants
Summary
Replaces the simple 0.9 embedding-similarity threshold for duplicate detection with an AI-powered semantic verification step. The Milvus embedding search now acts as a candidate filter (≥ 0.8 similarity), and the Cloud Agent verifies whether any candidate is a true duplicate using LLM reasoning.
Changes
Duplicate detection is now two stages instead of one
Cloud Agent supports prompt-only sessions
UI error sanitization