Migrating conversation translation to API2 BullMQ

Why

Translation used to be scheduled and executed by Rails GoodJob:

Rails producer (post-sync, QA prep, POST /conversations/:id/translate)
  → GoodJob TranslateConversationJob (queue: latency_30s)
    → Conversation#translate (inline OpenAI calls in Rails)

Once API2 owned the translation worker, the GoodJob job became a bridge that only POSTed to API2 — but it still occupied latency_30s capacity, the same pool that latency-sensitive work depends on, and it was gated by the api2_conversation_translation flag so two execution paths had to be kept alive.

After

Every producer hands the conversation straight to API2, and API2 owns the queue:

Rails producer → POST /internal/conversation_translations/enqueue (API2)
Browser        → POST /conversations/{id}/translate (API2, session cookie)
  → enqueueConversationTranslation()
      skip/no-op prefilter (multilingual off, known English, already translated)
      BullMQ conversation-translation job, deterministic job id
        → runConversationTranslationJob (translation state + credits + OpenAI)

Both routes go through the single enqueueConversationTranslation helper in src/lib/conversation-translation-enqueue.ts, so skip semantics, the deterministic job id (conversation_translation_<org>_<conversation>_<sourceVersion>) and translation-state writes are identical regardless of the caller.

The browser route

POST /conversations/{id}/translate on API2 replaces the Rails endpoint for the conversation view’s Translate button. It is deliberately not under /internal/*: it runs behind the normal authMiddleware session cookie check and authorizes the request the way ConversationPolicy#show? does —

• the Rails organization is resolved from the authenticated session, never from the request,
• loadMembershipScope loads the caller’s membership scope,
• the conversation lookup is filtered by conversationVisibilitySql, so a conversation outside the caller’s team/partner/group scope reads as 404.

The response keeps the semantics the UI needs: 202 with the resulting translation_state (in_progress for a queued translation), plus skipped / skipReason for conversations that need no translation work. The UI only invalidates the conversation queries on success, so the shape change from the Rails conversation blueprint is not user visible.

The Rails POST /conversations/:id/translate endpoint is removed: the app is its only caller, so translation is now requested from API2 only.

Rails side

• Conversation#translate_later calls Api2ConversationTranslationEnqueuer.enqueue! unconditionally. The api2_conversation_translation flag and the GoodJob fallback are gone, so current producers (post-sync analysis, QA evaluation request stage processor, and the user-triggered Translate action) create no new TranslateConversationJob row.
• The temporary historical call-translation migration used a paced Rails sweeper for already-transcribed calls with translation_state = pending. That sweeper has been removed; current translation producers use the API2 enqueue path directly.
• Conversation#translate and the legacy ConversationChannel translation:* broadcasts are left in place as the rollback path. Nothing schedules them.

TranslateConversationJob: retry bridge and drain path

The class is kept and reduced to a retryable bridge that hands the conversation to API2 (retry_on Api2EnqueueError, attempts: 10), because unfinished production good_jobs rows still need a constant to deserialize into. It is also moved off latency_30s (queue_as :default), which affects only retries of drained rows — existing rows keep the queue name they were enqueued with.

It is also the out-of-band retry path for post-sync analysis: Conversation#translate_later_best_effort (used by perform_post_sync_analysis) catches DeliveryError, reports it as a warning and enqueues this job. Without that, a transient API2 502/503/read timeout on the handoff skipped CX risk analysis for the conversation and made API2’s POST /internal/provider_conversation_syncs/complete call return 500, so the whole sync completion was retried for a translation-only failure. The browser route still handles its delivery errors itself, so it keeps the synchronous behaviour.

Deletion condition. TranslateConversationJob can only be removed once the post-sync best-effort path no longer needs an out-of-band retry vehicle, and

SELECT COUNT(*) FROM good_jobs
WHERE serialized_params->>'job_class' = 'TranslateConversationJob'
  AND finished_at IS NULL;

returns 0 in production with no scheduled retries remaining. Do not delete the production rows themselves; let them drain or age out. The legacy Rails translation code (Conversation#translate, translation:* broadcasts) can be removed independently.