ENG-2489: Improve error messages for misconfigured dynamic erasure email connectors

[JadeCara]

Ticket ENG-2489

Description Of Changes

When a dynamic erasure email connector is incorrectly configured (e.g., empty secrets), the error message currently shows a generic Pydantic validation error with no indication of which connector is broken. This makes debugging difficult for end users.

This PR improves error handling across the dynamic erasure email flow to always identify the misconfigured connector by key.

Code Changes

• Wrap DynamicErasureEmailConnector.get_config() to catch ValidationError and re-raise with the connector key in the message
• Handle per-connector errors in send_email_batch so one broken connector doesn't block others
• Always requeue successfully-processed privacy requests even when a connector fails
• Treat misconfigured connectors as needing email send in needs_batch_email_send to fail visibly rather than silently skip
• Add test for improved error message on empty secrets

Steps to Confirm

1. Create a dynamic erasure email connector with empty secrets
2. Trigger an erasure privacy request
3. Confirm the error message includes the connector key (e.g., Dynamic erasure email connector 'my_connector' is incorrectly configured: ...)
4. Confirm other correctly-configured connectors still process successfully

Pre-Merge Checklist

• Issue requirements met
• All CI pipelines succeeded
• CHANGELOG.md updated
  • Add a db-migration This indicates that a change includes a database migration label to the entry if your change includes a DB migration
  • Add a high-risk This issue suggests changes that have a high-probability of breaking existing code label to the entry if your change includes a high-risk change (i.e. potential for performance impact or unexpected regression) that should be flagged
  • Updates unreleased work already in Changelog, no new entry necessary
• UX feedback:
  • No UX review needed
• Followup issues:
  • No followup issues
• Database migrations:
  • No migrations
• Documentation:
  • No documentation updates required

🤖 Generated with Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

2 Skipped Deployments

Project	Deployment	Actions	Updated (UTC)
fides-plus-nightly	Ignored	Preview	Mar 31, 2026 6:24pm
fides-privacy-center	Ignored		Mar 31, 2026 6:24pm

[claude]

Code Review: Improve error messages for misconfigured dynamic erasure email connectors

The goals here are solid — surfacing the connector key in Pydantic validation errors, making send_email_batch process connectors independently so one failure doesn't block others, and routing misconfigured connectors into the visible error path. The approach is good, but there is one critical runtime bug introduced by the refactor.

---

Critical (must fix)

requeue_privacy_requests_after_email_send is called after the session is closed (see inline comment on line 118 of email_batch_service.py).

The with self.get_new_session() as session: block closes at line 114. Lines 115–119 are outside it, so on the success path, session is already closed and privacy_requests is a Query bound to that closed session. requeue_privacy_requests_after_email_send iterates the query and calls privacy_request.save(db=db) — this will raise a DetachedInstanceError at runtime on every successful batch send.

The fix is to move the if has_failure: check and the requeue_privacy_requests_after_email_send(...) call back inside the with self.get_new_session() as session: block. The original code had this in the correct place inside the try: block.

Note: the integration tests for the failure path mock requeue_privacy_requests_after_email_send, which means this regression won't be caught by those tests. Only a successful end-to-end run would surface it.

---

Suggestions

• Broad except Exception (inline on line 90): transient infrastructure errors are now silently absorbed instead of allowing Celery to retry. Worth documenting as a known trade-off, or narrowing the catch for certain error types.
• "third_party_vendor_name" assertion (inline on test_dynamic_erasure_email_connector.py line 290): couples the test to Pydantic's internal ValidationError formatting — consider removing it since the surrounding assertions already cover the intent.
• Missing test for needs_batch_email_send exception path: the new except Exception block in request_runner_service.py (lines 1174–1183) that treats check errors as "needing email" has no unit test. A test verifying that an exception during needs_email() still returns True would prevent regressions.

[galvana]

This is a nice set of improvements to the error handling. I spent some time going back and for with Claude about what would happen if email connector A succeeds and B fails. As I understand it, today we would just re-send the email for connector A the next time the email batch kicks off. This is a problem that existed before your current change so we don't have to fix it now, but it's something to think about.