Skip to Content
Internal docs are powered by Nextra Docs Theme.
SystemsPlatformAPI2Rails to API2 migrationPackagesNative conversation-QA completion

Native conversation-QA completion

ENG-2832  moves the correctness-critical completion phase after a native result import into API2. The package also replaces the Rails organization-context request with a tenant-safe Drizzle lookup and signed logo URLs.

Ownership contract

  • native_conversation_qa_result_import must accept the result run before native completion can own it.
  • native_conversation_qa_completion selects the completion owner for a new accepted import. Both flags are organization-scoped and fail closed.
  • api2.conversation_qa_result_imports.completion_owner records api2 or rails in the same transaction as the result import and evaluation claim. Retries use the recorded owner instead of reselecting it.
  • API2 completion handles success, skip and failure terminal states. Success completes imported agent evaluations, overrides prior active AI evaluations, calculates the parent score, completes the request, links the evaluation batch and refreshes conversation QA scores.
  • Failure marks pending agent evaluations and the request failed. Skip records the request and conversation skip reason. Contradictory terminal payloads are rejected before result import.
  • Billing jobs and conversation-summary work are queued only after the import transaction commits. Rails keeps the effects-only callback for result imports whose durable completion owner is rails.
  • Organization context resolves an exact slug before attempting a numeric internal ID, matching Rails’ friendly lookup and preventing ambiguous numeric slugs from crossing tenants.

Deploy and observe

Deploy PR #11312  in both regions and apply the Drizzle migration before changing flags. Verify the new column, default and non-null constraint, then enable completion only for an organization with clean native-import evidence.

For each cohort, record:

  • result-import counts grouped by completion owner and outcome;
  • finalization queue waiting, active, delayed and retained-failed counts;
  • successful, skipped and failed request and agent transitions;
  • parent and conversation score parity with the retained Rails path;
  • duplicate or conflicting result-run markers;
  • billing and conversation-summary enqueue failures;
  • unexpected calls to the Rails result or effects endpoints.

Require real terminal traffic in both regions and 24 clean hours at the final cohort before removing the Rails completion path.

The rollout gate passed on September 8, 2026 at 22:01 UTC. Result import and completion are enabled in both regions. The final audit found no marker conflict, duplicate result, retained finalization failure, or repeated effect. This passes production ownership; it does not remove the rollback flag or Rails effects path.

Rollback

Disable native_conversation_qa_completion first. New accepted imports then record Rails completion ownership. Existing API2-owned markers must continue through API2 on replay, so keep the native code deployed until they drain. If needed, disable native_conversation_qa_result_import afterward to return new runs to the full Rails writer. Do not edit markers, result-run IDs, evaluation state or queue payloads during rollback.

Owner: Platform. The code gate was introduced September 6, 2026, and the PostHog flag is enabled in both regions. Remove or review it by December 5, 2026.

Last updated on