UN-3232 [FEAT] Implement back-off retry mechanism with deadline-aware timeout budgeting

[muhammad-ali-e]

What

• Implement exponential back-off retry mechanism for transient HTTP errors (429, 5xx, ConnectionError, Timeout)
• Add deadline parameter to _send_request() to cap total retry time within a caller-specified budget
• Fix whisper() to use wait_timeout as the total time budget (POST + polling)

Why

• The Python client had no retry mechanism — transient errors (rate limits, server errors, network blips) caused immediate failures
• Without deadline-awareness, retries could multiply the effective timeout far beyond what the caller expects
• wait_timeout was misused as the raw HTTP timeout, so worst-case total time could reach ~900s when the user expected 180s

How

• Added max_retries, retry_min_wait, retry_max_wait constructor parameters to LLMWhispererClientV2
• Added _RetryableHTTPError wrapper, _is_retryable(), _log_retry(), _retry_wait() (with Retry-After header support for 429s)
• _send_request() uses tenacity with exponential jitter backoff, stop_after_attempt, and optional stop_after_delay via the deadline parameter
• whisper() now sets start_time before the POST, computes deadline = start_time + wait_timeout, and passes post_timeout = min(api_timeout, wait_timeout) so POST + polling stays within the user's budget
• Added pytest-mock dev dependency and comprehensive unit tests (15 retry tests + 4 deadline/timeout tests)
• Added CONTRIBUTING.md with development setup, testing, and contribution guidelines
• Added integration test configuration with conftest.py

Can this PR break any existing features. If yes, please list possible items. If no, please explain why.

• No. Retry parameters have sensible defaults (max_retries=3). The deadline parameter defaults to None, preserving existing _send_request() behavior for all callers except whisper(). The whisper() public API signature is unchanged — only internal timeout logic was corrected. All existing tests continue to pass.

Database Migrations

• None

Env Config

• None

Relevant Docs

• N/A (parameter docs live in docstrings within client_v2.py)

Related Issues or PRs

• UN-3232

Dependencies Versions

• Added pytest-mock ~=3.14 as dev dependency

Notes on Testing

• uv run pytest tests/unit/client_v2_test.py -v — all 19 tests pass
• Retry tests: connection error, timeout, 429, 500 retries; no retry on 400/401; exhausted retries; disabled retries
• Deadline tests: test_whisper_post_uses_min_of_api_timeout_and_wait_timeout, test_whisper_post_uses_wait_timeout_when_smaller, test_send_request_deadline_caps_timeout, test_send_request_deadline_stops_retries

Screenshots

• N/A

Checklist

I have read and understood the Contribution Guidelines.

[github-actions]

filepath	function	$$\textcolor{#23d18b}{\tt{passed}}$$	SUBTOTAL
$$\textcolor{#23d18b}{\tt{tests/integration/client\_v2\_test.py}}$$	$$\textcolor{#23d18b}{\tt{test\_get\_usage\_info}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{tests/integration/client\_v2\_test.py}}$$	$$\textcolor{#23d18b}{\tt{test\_whisper\_v2}}$$	$$\textcolor{#23d18b}{\tt{9}}$$	$$\textcolor{#23d18b}{\tt{9}}$$
$$\textcolor{#23d18b}{\tt{tests/integration/client\_v2\_test.py}}$$	$$\textcolor{#23d18b}{\tt{test\_highlight}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{tests/integration/client\_v2\_test.py}}$$	$$\textcolor{#23d18b}{\tt{test\_whisper\_v2\_url\_in\_post}}$$	$$\textcolor{#23d18b}{\tt{4}}$$	$$\textcolor{#23d18b}{\tt{4}}$$
$$\textcolor{#23d18b}{\tt{tests/integration/client\_v2\_test.py}}$$	$$\textcolor{#23d18b}{\tt{test\_webhook}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{tests/unit/client\_v2\_test.py}}$$	$$\textcolor{#23d18b}{\tt{test\_register\_webhook}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{tests/unit/client\_v2\_test.py}}$$	$$\textcolor{#23d18b}{\tt{test\_get\_webhook\_details}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{tests/unit/client\_v2\_test.py}}$$	$$\textcolor{#23d18b}{\tt{test\_whisper\_json\_string\_response\_error}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{tests/unit/client\_v2\_test.py}}$$	$$\textcolor{#23d18b}{\tt{test\_whisper\_json\_string\_response\_202}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{tests/unit/client\_v2\_test.py}}$$	$$\textcolor{#23d18b}{\tt{test\_whisper\_invalid\_json\_response\_error}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{tests/unit/client\_v2\_test.py}}$$	$$\textcolor{#23d18b}{\tt{test\_whisper\_invalid\_json\_response\_202}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{tests/unit/client\_v2\_test.py}}$$	$$\textcolor{#23d18b}{\tt{test\_retry\_on\_connection\_error}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{tests/unit/client\_v2\_test.py}}$$	$$\textcolor{#23d18b}{\tt{test\_retry\_on\_timeout}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{tests/unit/client\_v2\_test.py}}$$	$$\textcolor{#23d18b}{\tt{test\_retry\_on\_429}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{tests/unit/client\_v2\_test.py}}$$	$$\textcolor{#23d18b}{\tt{test\_retry\_on\_500}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{tests/unit/client\_v2\_test.py}}$$	$$\textcolor{#23d18b}{\tt{test\_no\_retry\_on\_400}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{tests/unit/client\_v2\_test.py}}$$	$$\textcolor{#23d18b}{\tt{test\_no\_retry\_on\_401}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{tests/unit/client\_v2\_test.py}}$$	$$\textcolor{#23d18b}{\tt{test\_retries\_exhausted\_raises}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{tests/unit/client\_v2\_test.py}}$$	$$\textcolor{#23d18b}{\tt{test\_retries\_exhausted\_500\_returns\_response}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{tests/unit/client\_v2\_test.py}}$$	$$\textcolor{#23d18b}{\tt{test\_retry\_disabled}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{tests/unit/client\_v2\_test.py}}$$	$$\textcolor{#23d18b}{\tt{test\_whisper\_post\_uses\_min\_of\_api\_timeout\_and\_wait\_timeout}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{tests/unit/client\_v2\_test.py}}$$	$$\textcolor{#23d18b}{\tt{test\_whisper\_post\_uses\_wait\_timeout\_when\_smaller}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{tests/unit/client\_v2\_test.py}}$$	$$\textcolor{#23d18b}{\tt{test\_send\_request\_deadline\_caps\_timeout}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{tests/unit/client\_v2\_test.py}}$$	$$\textcolor{#23d18b}{\tt{test\_send\_request\_deadline\_stops\_retries}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{tests/utils\_test.py}}$$	$$\textcolor{#23d18b}{\tt{test\_redact\_key\_normal}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{tests/utils\_test.py}}$$	$$\textcolor{#23d18b}{\tt{test\_redact\_key\_different\_reveal\_length}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{tests/utils\_test.py}}$$	$$\textcolor{#23d18b}{\tt{test\_redact\_key\_non\_string\_input}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{TOTAL}}$$		$$\textcolor{#23d18b}{\tt{38}}$$	$$\textcolor{#23d18b}{\tt{38}}$$

[jaseemjaskp]

LGTM