Rails to API2 migration archive

Archived October 3, 2026. Rails has been removed from the repository and deployment infrastructure. API2 is the only application backend. The rollout flags, Rails fallbacks, rollback commands, and remaining-work statements below are historical evidence and must not be used as current operating instructions.

Final state

The migration is complete. Rails source, services, local setup, runtime clients, fallbacks, and ownership switches have been removed. The documents in this section are retained only as a decision and rollout record. For current development instructions, use the repository AGENTS.md and rulebase-web/rulebase-api2/AGENTS.md files.

See the Rails runtime retirement archive for the final removal record.

Historical migration record

September 17 canonical API2 gateway milestone

Goal 1 shipped in both production regions on September 17. api.rulebase.co, eu.api.rulebase.co, and their api2.* equivalents now serve API2. Rails serves its remaining contracts through unadvertised regional legacy-rails.* origins. Native API2 routes take precedence, and unmatched legacy HTTP contracts are forwarded internally. Existing public provider and OAuth URLs remain valid, so the cutover does not require blanket Intercom, Zendesk, Jira, Slack, or OAuth reconfiguration.

The canonical API2 gateway package records the audited live route surface, proxy requirements, deployed regional domains, acceptance and rollback evidence, and the separate Action Cable-to-SSE retirement plan. The legacy origins remain publicly routable but absent from customer docs, generated URLs, and UI configuration. Restricting them is separate hardening; it is not a Goal 1 gate. Goal 2 still requires removing all API2-to-Rails calls, compatibility traffic, Action Cable, and retained GoodJob work before Rails can be deleted.

September 9 Rails table ownership map and legacy dashboard retirement

The Rails table ownership map classifies all 166 public.* tables declared in rails-schema.ts (read-only projection, API2 writer, Rails writer, dual-written, or retirement candidate) and sequences the work to decouple API2 from Rails persistence. The first slice ships here: the dead public.dashboard_* instance tables are renamed to deprecated_dashboard_* (Rails removed its dashboard code in #7095; API2 has used api2.dashboard_* exclusively since drizzle 0060), their dead declarations are removed from rails-schema.ts, and pnpm db:verify-table-retirement now audits table retirement both in code and in the database.

September 9 weekly agent performance reports preparation

The weekly agent performance reports package moves the full weekly report pipeline to native API2/BullMQ: the daily 08:00 UTC generation tick, the Monday 09:00 UTC preview plus 30-minute auto-send, per-organization fan-out, report lifecycle, native OpenAI content generation, citation persistence, and Resend email delivery — with no Rails /internal/* domain endpoints executing the work. Deterministic job IDs, persisted job.updateData step markers, per-report outbound_delivery_claims, and the emails_already_sent_for_period gate make every retry resume without duplicate reports, content, citations, or notifications. The two retired Rails crons stay registered but enabled_by_default: false under the pause_api2_agent_performance_reports kill switch; generate_later, admin actions, and rake tasks enqueue through the API2 /enqueue* routes.

This is code preparation only. Ownership requires deployment in both regions, one clean daily generate tick and one Monday preview/auto-send, zero new normal-path GoodJob rows for the five retired classes, and 24 clean hours; the runbook lists the observation queries and the non-racing rollback.

September 8 native QA related-conversation refresh

ENG-2731 moves the normal kept/supported sync_related_conversations stage from Rails’ synchronous preparation endpoint into API2. The native stage keeps the request lease and Rails thread ordering, checkpoints one conversation per call, and persists a stable provider-sync run ID before external work so an ambiguous response replays the same outbox. Missing sources and provider failures remain best-effort; tenant/source mismatches fail closed.

Unsupported, discarded or timestamp-incompatible sources deliberately retain the Rails primitive while their production population is censused. This is a bounded QA-stage ownership transfer, not full QA completion. Captioning, eligibility/sampling, evaluation creation and terminal failure effects remain on the critical path.

This section is the source of truth for migrating Rulebase application traffic, data ownership, and background workloads from Rails to API2. It complements the detailed API2 runbooks already in this section; it does not replace them.

September 8 implementation milestone and runtime traffic

PR #11533  merged to main at 7111e1ac219f9f0b65b923ac12c0f998369da30f with all four API2 test shards, aggregate CI, build, formatting, Codecov, and automated review green. This clears the September 8 code/parity-readiness gate for the scoped retained contracts. It does not clear the separate GoodJob-zero or Rails-retirement gates.

Comparable Rails worker logs show combined executions falling from an August 1–7 average of 1,885,736 per day to 239,684 per day on September 1–7, an 87.3% reduction. The median fell 91.1%. The roughly 34,000 EU and 16,000 US jobs shown in GoodJob on September 8 cover only its retained 12-hour window. Most remaining application volume is Zendesk and Jira webhook transport, followed by CX risk, dispute acknowledgement, and completion residue. Active Storage analysis, CloudWatch publication, autoscaling, and heartbeats are Rails-operational work and are excluded from application migration priority.

See the GoodJob traffic baseline and retirement priorities for the exact historical series, current regional class counts, webhook relay design, active implementation handoffs, and runtime completion gate.

September 7 native WorkOS event processing preparation

ENG-2858 moves the ten-minute WorkOS Events poll and its retry budget from Rails GoodJob to API2’s maintenance BullMQ queue. API2 preserves the complete 17-type filter, fetches raw snake-case JSON, persists and locks each event, and applies organization, identity, membership, invitation and RBAC effects before advancing the shared cursor.

The Rails cron entry is removed in the candidate release, while the job class and event-effect implementations remain dormant for queued work and code rollback. Production ownership requires deployment in both regions, one scheduler per region, actual event evidence, no new Rails recurring execution or callback, no retained failure or duplicate effect, and 24 clean hours.

September 7 native headless authentication

ENG-2842 moves the email-first WorkOS feature probe, CSRF setup, Magic Auth, password login/signup, email verification and password reset plus their first-party client from Rails to API2. The native path uses private Rack session IDs, Rails-compatible persisted masked CSRF, atomic session rotation, native account/membership provisioning, exact Origin checks, bounded Redis rate limits, generic credential errors and enumeration-safe reset requests.

The existing global headless_auth Flipper gate remains the fail-closed kill switch: off or unavailable selects hosted AuthKit. Rails keeps the headless actions for code-revert rollback during observation. Production credit requires regional deployment, actual login/signup/verification/reset evidence with no Rails headless request, valid downstream session/logout behavior and 24 clean hours. No provider, flag, data, queue or schema mutation is part of the package.

September 7 native QA completion and evaluation-check bodies

ENG-2832 makes the terminal phase after a native conversation-QA result import executable in API2. API2 can now update agent and parent evaluation state, request state, override lineage, workload links and conversation scores in the same transaction as the import. A durable completion_owner marker keeps every accepted result run on its original completion path across retries and later flag changes. Billing and conversation summary work remain asynchronous after the transaction.

ENG-2831 adds native complaint, customer-issue and sales-call compliance bodies to the existing evaluation-check BullMQ delivery path. Each type has an independent organization-scoped PostHog flag. Disabled, unsupported or unavailable flag evaluations continue through the serialized Rails executor.

PR #11312  merged at 59c615b15e7; deployment alone transfers no ownership. All four new flags are default-off and fail closed. Production credit requires regional deployment, staged organization/type enablement, real success/skip/failure evidence, bounded fallback and retained-failure counts, and 24 clean hours. The earlier native_conversation_qa_result_import flag remains a prerequisite for native completion.

September 6 Rails schema-binding compatibility

ENG-2682 makes Rails’ four API2-owned model bindings independently selectable for the public-schema cutover. The empty RULEBASE_API2_PUBLIC_RELATIONS default preserves the current api2 tables; after a reviewed physical move, operators can name bot_providers, screening_searches, status_incidents, organizations or work_items without waiting for another application build. Unknown values fail closed, and the status-incident join uses the same mapping for the collision-safe public organization name.

The operational query-data ACL probe now resolves the future public agent view first and falls back to the current agent_api view. Together with the merged assignment-routine compatibility package, executable cutover blockers fall from 20 to nine; all nine remaining references are deliberately confined to the work-item-resource retirement script until old workers drain. This is application preparation only. No schema move, environment selection, grant, queue, data or runtime-ownership change occurs in the package, and productionCutoverReady remains false.

September 6 WorkOS authorization reconciliation preparation

ENG-2815 moves the daily 03:15 UTC permission, environment-role and organization-role import from Rails/GoodJob to API2’s maintenance BullMQ queue. The native path preserves assignable/widget filtering, durable WorkOS mappings, collision-safe tenant role keys, additive permission behavior and per-item/per-organization failure isolation with aggregate retry.

This is code preparation only. Production ownership requires deployment in both regions, one actual 03:15 run, local/WorkOS parity, no retained failure and 24 hours of observation. WorkOS events, remote writes and role CRUD remain separate AUTH dependencies.

September 6 evaluation-check delivery preparation

The evaluation-check run package moves the normal Evaluations::RunEvaluationCheckJob producer, retry budget and terminal failure delivery from GoodJob to a deterministic API2 BullMQ queue. At this point Rails kept the tenant-scoped complaint, customer-issue and compliance implementations behind authenticated synchronous endpoints, and immediate API2 delivery failure fell back to the retained GoodJob class. A separate eight-attempt failure queue serialized the terminal transition with execution through the same per-check PostgreSQL advisory lock. The September 7 native-body package supersedes that implementation boundary without changing the delivery contract.

This is code preparation only. Production ownership requires deployment in both regions, real checks for every traffic-bearing type, bounded fallback and retained-failure counts, no stranded snapshots or duplicate effects, and 24 hours of clean observation. Rails retirement remains a later package.

September 7 assignment-interval schema cutover

ENG-2682 migration0344 is the first deployable domain move from the pinned public-schema plan. It moves the assignment-interval table, owned sequence, primary-key index and physical rebuild function to their now-unoccupied canonical public names without changing their OIDs, while retaining an updatable table alias and routine wrapper under api2 for rolling deployments. Fresh US/EU catalog, owner, ACL/default-ACL, routine-body, target-collision, index-health and lock/load checks pass. The exact migration and reverse transaction pass the PostgreSQL 15.10 behavior rehearsal. Deployment and observation remain open; the broader schema manifest is still preparation only.

September 6 regional capture reconciliation

Fresh ~15:00 UTC ECS-exec catalog/ACL captures from both regions (US PostgreSQL 15.17, EU 15.14) show clean identity bridges — 0 missing numeric mappings, missing legacy rows, slug mismatches or duplicate numeric-mapping groups — and the assignment-interval routine at immutable migration0336 in both regions. The routine rehearsal and pinned evidence were re-targeted from migration0326 to 0336 and rerun green. The manifest gained the three live ENG-2775 bot-topic escalation tables and six live foreign keys (197 tables, 719 indexes, 373 affected foreign keys per region); the four api2.work_item_resource* tables remain manifest-planned but deploy-pending, so live shows 193 tables and 364 foreign keys. Live production has already dropped the legacy public.conversation_assignment_intervals collision (28/26 live vs 29/27 pinned). Executable writer blockers drop from 23 to 20 after three dynamic bindings were made schema-agnostic. productionCutoverReady remains false.

September 6 current-main schema rehearsal

ENG-2682 now matches source revision 0a076d5d639: 194 API2 tables, 12 sequences, 716 indexes and 367 API2-owned or API2-referencing foreign keys per region have reviewed public destinations. Fourteen tables added since the September 2 snapshot are included, including the native QA result-import marker merged while this package waited for review. The new Rails public.conversation_assignment_intervals object also changes the native table, owned sequence and primary key into explicit collisions, bringing the planned totals to 29 table-name and 27 index-name collisions.

The PostgreSQL 15.10 forward/reverse rehearsal passes for all 193 tables present in the checked-in Rails structure, 76 checked-in agent views, 12 sequences and 1,451 total foreign keys. The new Drizzle-owned result-import table is planned but reported as missing from that older structure snapshot. It now rejects all unplanned relations/index collisions at once, validates the 367-FK manifest exactly and accepts only the reviewed assignment-routine signature. The exact-main writer scan finds 23 executable references requiring disposition; nine newly surfaced references belong to the guarded legacy screening-resource retirement script and remain gated on old workers draining. This is cutover preparation only. Fresh US/EU catalogs, ACL/default grants, identity parity and lock/load budgets remain required, so production readiness is still false.

September 5 WorkOS membership reconciliation preparation

ENG-2809 moves the nightly WorkOS organization-membership scan from Rails/GoodJob to API2’s durable maintenance queue at the same 03:00 UTC cadence. The native task pages every mapped organization, reconciles kept memberships by WorkOS ID, imports missing roles with only Rails-assignable permissions, and retries idempotent whole-run attempts when any organization fails.

This is code preparation only. Rails retains the job class for queued work and code-revert rollback, but its recurring cron entry is removed in the candidate release. Ownership requires both regional schedulers, an actual daily run, membership/role parity, no retained failures, and 24 hours of observation.

September 5 evaluation-schedule ownership preparation

ENG-2808  prepares the 15-minute evaluation-schedule scan and durable per-window retries to move from GoodJob to BullMQ. API2 selects a bounded, stable set of due schedules and enqueues one deterministic job per schedule/window. Rails temporarily retains only the tenant-scoped batch-construction transaction behind an authenticated synchronous endpoint.

This is code preparation only. Production ownership requires review, deployment in both regions, real scheduled windows, no duplicate batches, and a clean 24-hour queue/error observation. Reverting the revision restores the retained Rails cron and job implementation; do not delete queued BullMQ work.

September 5 CX-risk completion fan-in

ENG-2249  moves the normal CX-risk snapshot completion signal and polling loop from Rails GoodJob to a deterministic BullMQ queue. Rails retains the tenant-qualified finalization endpoint and its existing database/model effects; a per-snapshot advisory lock makes ambiguous delivery and legacy fallback overlap safe. This is code-ready only. See the package runbook for deployment, observation, and rollback gates.

September 5 provider post-transcription continuation preparation

ENG-2803  and ENG-2962  and PR #11223  move the normal provider-sync post-transcription decision and fan-in into API2. API2 now selects eligible call and image work, applies the Rails-compatible organization throttle, rebuilds the transcript after the deterministic BullMQ fan-in, marks terminal transcription state, and enqueues translation. Rails retains only the narrow CX-risk dispatch and an explicit translation-delivery fallback.

This is code preparation only. API2 now applies the responder, Jira customer field, and stale-ticket guards and enqueues CX-risk starts directly. Rails retains a narrow translation-fallback endpoint plus the broad analysis and completion endpoints for rolling deploy and code-revert rollback. No production ownership or migration credit changes until the exact revision is deployed in both regions and passes queue, transcript, translation, CX-risk, fallback, and Rails-traffic observation gates.

September 4 direct Intercom ingress preparation

ENG-2800  prepares API2 to receive Intercom’s single production webhook destination directly. An externally signed US delivery is synchronously copied byte-for-byte to EU API2 with internal authentication before US processing; authenticated regional copies never fan out again. A failed required EU copy returns a retryable error before local effects, preserving the existing Rails ordering.

This is code preparation only and adds no migration credit. Rails remains the configured provider destination until the PR deploys in both regions, the direct API2 route is probed, and an explicitly authorized Intercom destination change is observed with real regional events. Keep the Rails URL as the immediate provider rollback until the clean observation gate passes.

September 4 schema writer inventory

ENG-2682 now has a deterministic repository-wide literal-reference inventory for 271 non-index manifest relations and the api2.rebuild_conversation_assignment_intervals routine. It classifies 619 active runtime and operational references, including 149 ordinary writes, and isolates 14 executable cutover blockers: seven table bindings, six assignment-routine caller references and one ACL probe. It found no active named-constraint upsert or direct sequence call. Dynamic SQL, external clients and 177 relations without an active literal reference remain manual evidence gates. Fresh regional schema/ACL exports are blocked until AWS SSO is renewed, so production cutover readiness remains false.

The September 6 dual-schema caller package closes the six routine references: Rails Jira/Zendesk sync and API2 provider persistence/backfill now prefer the future public signature and fall back to the current api2 signature. The old wrapper must remain through physical movement and rollback observation; table bindings, the ACL probe and the remaining production cutover gates stay open.

September 3 native authentication foundations

ENG-2741 native logout merged in #11073 at 10:46 UTC. Regional WorkOS settings matched Rails at 10:40; the containing release succeeded in EU at 11:05 and US at 11:06. Both runtimes expose the native contract and required configuration. Actual logout/revocation/redirect events and the 24-hour observation gate remain open.

ENG-2751 organization switching merged in #11091. Its API2 server deployed in both regions through release e74224443f; GitHub Actions run 33789129052 deployed that exact revision to both regional UI Workers, so the caller is live and actual-event/24-hour observation remains open. ENG-2758 hosted login has server compatibility deployed in both regions, and the user confirmed both regional API2 callback URLs are registered in WorkOS. The first-party hosted login callers are now staged to move to API2 with Rails-compatible enterprise SSO connection selection. Invite acceptance and identity lifecycle remain explicit AUTH dependencies. September 4 foundations and September 15 retirement remain open; these bounded packages do not establish a full domain closure.

Work-schedule server compatibility

ENG-2742 adds native create/update/discard contracts and a transaction lock shared with Rails default selection. ENG-2752 moved the first-party callers in #11093. The server revision deployed through release e74224443f, and GitHub Actions run 33789129052 deployed that exact revision to both regional UI Workers. Local contract/concurrency proof is recorded in the package runbook; actual mutation events and production observation remain open.

Target architecture

The program-level end state is tracked in ENG-2681 : API2 is the sole application backend, BullMQ is the sole asynchronous runtime, and the rulebase-api Rails project and all Rails/GoodJob deployment infrastructure are deleted. After Rails retirement, api.rulebase.co and its EU equivalent become the canonical native service host; api2.* remains a compatibility hostname rather than being removed. External public contracts are consolidated under versioned /v1/... paths. Prefer both hostnames serving the same application over blanket redirects, because redirects are unsafe for some non-GET clients.

Before that deletion gate, the canonical API2 gateway milestone makes API2 the sole public ingress while Rails remains an access-restricted compatibility origin. Passing that milestone does not count forwarded Rails routes, Action Cable, or API2-to-Rails calls as retired.

That end state has five gates:

1. Every required webhook, including Intercom and Zendesk, terminates and retries in API2 without Rails forwarding or reconciliation ownership.
2. Every required background job, cron, callback, fan-out, and fan-in is API2/BullMQ-owned, and GoodJob has no normal-path traffic.
3. The first-party app dual-loads Rails and API2 while synchronous route, authorization, side-effect, and response parity closes, then cuts fully to API2.
4. PostgreSQL converges on the public schema only through ENG-2682 ; both the api2 and agent_api application schemas are consolidated through new roll-forward migrations before removal. Extension and system schemas remain separately inventoried.
5. Production traffic, external clients, DNS/load balancers, callbacks, schedules, operational tooling, and rollback paths have zero Rails dependency before the Rails project is deleted.

September 15 delivery target

Roadmap dates are latest target milestones, not start dates. Pull ready work forward immediately. The user approved starting the September 3 foundations on September 2: native auth first, bounded QA stage/failure ownership second, and schema compatibility preparation and rehearsal third. Finish complete, tested, reviewable capabilities and keep advancing within the current six-workstream limit.

This is an ambitious target, not a forecast proved by current throughput. Surface any missed critical-path gate immediately with its exact dependency and recovery sequence. Keep the full required validation and observation window.

Session retention ownership

ENG-2738 transfers the midnight session cleanup scheduler and bounded execution to API2. Existing-session validation is already native; WorkOS/login/logout and organization switching remain separate AUTH dependencies. Retention’s actual midnight and 24-hour regional observation gates remain open. See the ownership inventory and rollout gates.

Current planning baseline

The completed September 2 source census at 1baf2de49054e46b80b1ab2f7e130a916b426964 identifies 1,041 Rails route records, 414 literal Rails client calls, 262 job/support files and 40 configured cron entries. Both regions have 180 API2 tables with 28 public-name collisions; the agent query surface has 78 US and 77 EU views. These are source/catalog counts, not measured traffic or completion percentages. Maintain incremental changes from current main instead of repeating the census.

There are 24 inventoried domains. Chidi retired the simulation product, completing that domain by product disposition and leaving 23 migration domains. Do not port simulation. Remove its obsolete callers, routes, subscriptions, reminder cron and queued/retrying work under OPS/REALTIME by September 11. Shared call analysis, call-script/assessment work, phishing candidates and native coaching sessions remain in retained scope.

The detailed census, roadmap and package register are in migration-planning/2026-09-02/ in the coordinated planning worktree. Runtime retirement certification and external-client confirmation remain execution gates. The original one-month target and 8–10 weighted-points/week goal are superseded. Legacy 64%/84% figures are historical; measure completed ownership gates, actual Rails dependencies removed and observed time to land/deploy/verify instead.

QA credit reconciliation #11039 and tag writes #11040 merged at September 3 00:39 UTC; retention #11044 also merged and is included in main 22fa7bcf28. Their deployment and observation gates remain separate from implementation.

Native auth foundation

ENG-2730  implements existing-session validation directly in API2, including persisted revocation, expiry and tenant permissions on every cache hit. The package runbook records compatibility evidence and rollout gates. Login/logout issuance, WorkOS lifecycle and Rails session-retention ownership remain follow-ups. This is merged in #11048  at September 3 01:44 UTC (64198ab5d4); deployed and observed status remain separate.

The subsequent production-regression task found an invalid accounts.discarded_at predicate, causing authentication server errors. A separate owner is repairing the query/fixture. Auth recovery must be observed before further runtime migration rollout; green fixture tests alone did not establish compatibility with the production account schema.

QA lifecycle and schema foundation preparation

A separate assignment-interval routine rehearsal now uses immutable migration0326 whose function body matches both regions at September 3 11:38 UTC. Seven actual PostgreSQL checks prove old-alias writes, native body/wrapper compatibility, alias removal and lossless reversal. Full regional schema/ACL restoration and current-manifest readiness remain open; no production objects moved.

ENG-2682 now also rehearses the complete repository database structure locally: 178 native tables, 76 agent views and 11 sequences move and reverse while 1,425 foreign keys and post-move writes survive. Six regional manifest objects and production grants are absent from that snapshot. Fresh regional schema/ACL restoration and writer readiness remain September 4 foundation dependencies.

The full matrix also plans every restored unique-index target: 323 indexes, including 18 partial indexes, pass before/after/reverse movement. All 192 named constraint targets fail through view aliases; 26 renamed constraints require new names even on physical tables, and 11 explicit old sequence names disappear. Runtime writer classification and row/trigger parity remain open.

ENG-2739 repairs the retained related-conversation adapter: bound native calls to one record, persist attempted IDs, and isolate provider SQL failures with savepoints while retaining lease fencing. Forty-eight focused Rails examples pass; regional deployment and actual-stage recovery remain open. This does not close native QA preparation.

ENG-2731 transfers stage selection/completion, persisted leases and atomic terminal failure to API2, with a durable public outbox for retained finalization effects. Customer relationship detection and the preparation-complete state transition now execute natively under the persisted lease, as do kept supported related provider refreshes. ENG-2885 advances ordinary conversation-part image captions natively and checkpoints failed/no-op candidates. ENG-2887 adds direct attachment caption selection, cache reuse, persistence and redaction tombstoning. Rails still owns conversation-part caption context/result persistence, eligibility/sampling, evaluation creation and result finalization. ENG-2766 prepares native live billing with a deterministic BullMQ job, atomic billed-marker/ledger persistence and a retained GoodJob delivery fallback; it still needs regional deployment and observation. These packages are partial foundation progress and do not yet pass the full QA Rails-unavailable gate. See rulebase-web/rulebase-api2/docs/qa-request-lifecycle.md.

The deterministic QA result-delivery queue now separates inference from result delivery, so a transient finalizer failure no longer repeats the model run. ENG-2801  adds the next bounded ownership slice: API2 validates the tenant-qualified result tree, imports criterion/check/scorecard/agent aggregates and atomically claims the result-run UUID with a durable ownership marker. Rails is narrowed to an effects-only callback for completion, sampling, notifications and billing. The PostHog rollout is fail-closed and defaults off; native lifecycle effects and regional production evidence remain required before Rails-unavailable QA can pass. See rulebase-web/rulebase-api2/docs/qa-evaluation-result-finalization.md.

ENG-2682 fixes public destination names and identity bridges and rehearses a representative forward/reverse move on local PostgreSQL 15.10. All seven checks pass, including tenant ACLs, sequences, foreign keys and post-cutover writes. Full schema/ACL restoration, writer classification and regional mapping gaps remain prerequisites for deployable table moves. See rulebase-web/rulebase-api2/scripts/schema-compatibility/README.md.

Five-stream operating model

The September 2 user authorization permits scoped migration and migration-regression merges after required checks, required reviews and package rollout criteria pass, including main’s automatic production deployment. The coordinator serializes merges. No review bypass, direct deployment, source/data repair, replay, flag/queue change or infrastructure change is authorized by that standing instruction.

Keep at most three workers and three active packages across migration tasks, including CI/review/deployment waits. Work directly; subagents are not required and are not spawned without an explicit request or applicable instruction. Finish or explicitly park non-actionable waits with an owner, next check time or event, and unresolved gate before reusing their slots. Five streams and three roadmap queues do not create extra agents.

Each run must advance a safe ownership package or concrete critical prerequisite toward a tested, reviewable outcome. Continue to the next ready package when capacity permits; do not stop at an audit, a small PR or an arbitrary per-run quota while useful authorized implementation remains. Reuse decisive tests and the daily production audit baseline. Keep incident dependencies local so independent implementation continues. Required CI, reviews, rollout criteria and production authorization boundaries remain unchanged.

Health and decisions

• September 2, 23:04–23:05 UTC: provider wait/active and delayed Zendesk intake were zero in both regions (two delayed provider jobs each). All eight services were at desired capacity. Both regions had zero overdue billed QA mirrors; EU org 41 had 27 charges in the last 24 hours, all mapped and mirrored, while US had no charges in that window. US Freshdesk source 25 advanced its checkpoint to 23:00 UTC; EU source 25 remains unmapped with no checkpoint. Freshdesk null Sentry issue last occurred at 19:45 before the fix finished deploying. Historical recovery and the EU coverage decision remain separate. Intercom deletion #11030 is merged; successful regional deployment statuses at 21:23 UTC do not prove deletion-event bake. No weighted points are credited.

• September 2, 19:38 UTC: targeted US/EU queries found no billed-but-unmirrored QA evaluation older than 30 minutes. EU provider wait fell to 3,987 and sampled oldest age to 216 minutes, while delayed webhook intake rose to 2,305. This is an incomplete bake, not resolved congestion. Freshdesk org/source 31/25 is mapped in US and unmapped in EU; both still lacked a checkpoint before the next scan. #11026 is merged but its exact production revision is not yet verified. Historical recovery after the retained 17:00 UTC Rails scan and EU coverage require separately scoped production authorization.

• September 2, 19:04 UTC: ENG-2707 is deployed in both regions. EU provider waiting work fell from 9,939 to 5,703, but all 16 slots remained active and the oldest sampled wait increased from 159 to 230 minutes. The new global webhook cap is four; 1,906 EU webhook jobs were delayed and US remained at zero provider wait. The US Freshdesk source scan is repeatedly failing on nullable legacy sync_data; the scoped follow-up normalizes SQL NULL and JSON null before checkpoint reads and writes. The EU legacy Freshdesk source has no API2 organization mapping and remains excluded; its migration prerequisite is unresolved. No production state was changed. This audit predates the five-stream authorization; its former program-wide stop is superseded by package-specific incident gates above.

• September 2, 15:05 UTC: EU provider-sync waiting work grew from 9,033 at 14:16 to 9,939, with all 16 global slots occupied; the oldest sampled wait was 159 minutes. Webhook intake shares the five-requests/second source limiter and previously allowed up to 160 active jobs across five replicas. ENG-2707  bounds intake fleet-wide and defers only the same tenant/source while its downstream sync is active in a congested queue. This mitigates amplification; the original backlog predates the executor deployment and is not fully explained. Provider expansion dependent on this limiter remains gated; independent code work continues under the later five-stream model. No production mutation, deployment, or rollout change was performed by this audit.

• ENG-2696  prepares recurring Freshdesk imports for API2. The 15-minute source scan and per-ticket import retries move to BullMQ; native provider sync remains the downstream owner. This code-only package does not change the recorded 64% production / 84% code-ready aggregate. Regional source mapping, queue capacity, deployment, and a 24-hour observation window remain prerequisites for production credit. See the Freshdesk rollout contract.

• Zendesk source-owned sync is globally enabled in US and EU. The requester foreign-key failure was fixed in PR #10734 , and the two expired/demo connections found during the audit were discarded.

• Rails still spent 8.1 worker-hours across 57,333 ProcessZendeskWebhookTicketSyncJob spans in the latest Sentry day, up from 6.1 hours and 29,333 spans in the prior window even as p95 improved from 1.80 seconds to 1.12 seconds. PR #10960  moves the coalesced ticket import and source-sync handoff to BullMQ while retaining Rails webhook receipt, its durable event row, and terminal fallback.

• Conversation transcription is enabled for 100% of organizations. Known undersized ElevenLabs inputs now terminate without futile retries through PR #10733  and the preflight added in PR #10739 .

• Residual direct call-detail transcription from PR #10913  is deployed in both regions. The latest retained API2 windows contained 986 US and 157 EU successful call jobs, with no new failures or backlog. Three Rails delivery fallbacks contained HTTP 404s during the API2 rolling deploy, with no later matching event; the 24-hour bake window remains open.

• Intercom conversation imports, ticket imports, webhook forwarding, and API2 webhook ingestion became globally enabled at 2026-08-30T22:29:50Z, and the four selection branches have since deployed in US and EU. The August 31 audit found 405 Rails forwarding failures, primarily timeout bursts during rolling API2 deployments. The recovery keeps direct API2 delivery as the normal path and uses the existing tenant-scoped Rails/GoodJob path on delivery failure.

• The forwarding recovery deployed to both regions on August 31. All five matching Rails Sentry issue groups had zero events in the latest 24 hours, and the mapped reconciliation sources advanced their watermarks through the September 1 audit.

• The reconciliation scanner also selected three legacy regional Rails sources whose organizations have no API2 mapping. Those sources cannot complete in API2 and produced retained retries every 15 minutes. The source-selection guard in PR #10888  is deployed in both regions and has produced no post-deploy failures; require a complete clean 24-hour window before cleanup.

• The separate legacy Rails Intercom drift-reconciliation cron became stuck on Kuda’s August 24 watermark in US. Its all-pages-in-one-job implementation repeatedly exhausted the 4 GB worker and blocked every later cron run. Its bounded, per-source, page-checkpointed API2 replacement is deployed, and the existing watermark is deliberately retained for overlap and rollback.

• Stable 100% paths should be baked into code before starting lower-confidence migrations. Real delivery-error fallbacks remain even when rollout-selection branches are removed.

• QA sampling recovery is globally enabled and healthy in both regions. API2 completed 1,766 retained recovery jobs in the latest regional 12-hour windows with no failures or backlog, while Rails still ran 48 duplicate scheduling sweeps per region. PR #10920  removes that duplicate Rails scheduling and makes API2 the permanent owner.

• QA evaluation request processing first split out the safe user-requested lane in PR #10926 . PR #10941 , tracked in ENG-2680 , makes scheduled and admin preparation API2-owned, serializes active-row creation per conversation, allows legacy duplicates to reach a terminal state, and adds an approval-only per-organization repair task. The follow-up partial unique index is deployed, valid, ready, and unique in both regions. Read-only checks at 2026-09-06 17:21–17:22 UTC found zero ticket-closed active requests older than 15 minutes and zero conversations with duplicate active requests in US and EU. Ticket-closed ownership was then enabled for EasyPay in both regions and Nala in EU at 17:30–17:31 UTC. The US Rho actor was corrected from rho (organization 18) to the real production tenant rho-tnkw (organization 19) at 2026-09-07 00:23 UTC, and the unintended US Nala actor was removed seven minutes later. Clean regional preflight then allowed US kuda-pilot and EU Qonto to enter the rollout at 00:33–00:35 UTC. The final actor sets are US rho-tnkw, kuda-pilot, and EasyPay, plus EU Qonto, EasyPay, and Nala. The production ownership gate passed September 8 at 22:01 UTC: the final audit found no overdue or duplicate active request and no new terminal queue failure. One Kuda request stalled during the pre-recovery worker replacement; PRs #11508 and #11512 repaired the lifecycle, and the incident did not recur after the corrected workers were live.

• The first post-deploy audit found a separate EU result-import regression. The deterministic PII override from PR #10614  replaced API2’s accepted result with an invalid failed check when a regular fail option had no justification choices. PR #10933  and PR #10935  reverted the override and its dependent follow-up. The release containing both reverts deployed successfully to US and EU at about 2026-09-02T00:42Z. Sentry’s last matching event was at 00:27:53Z, before the recovery deploy; ten US and nine EU ticket-closed requests completed afterward. Keep expansion gated through the full observation window and stale-row repair.

• API2 provider sync is retrying one active legacy Zendesk source whose stored configuration has no subdomain. The queue was current and Rails fallback protects terminal failures, but neither runtime can safely infer the missing account identity. Repair it only through an approved, tenant-scoped production-data operation; do not suppress or globally skip these syncs.

• Workspace search has been at an untargeted 100% PostHog rollout since August 21. The project does not retain feature-flag evaluation events, so direct call-volume evidence is unavailable; the bake-in therefore requires an explicit post-deploy search and Sentry window before flag deletion.

• Manual exports have been at an untargeted 100% rollout since August 21. Two fresh EU manual QA workbooks completed successfully, both regional queues were empty in the latest successful regional audit, and the deterministic harness now matches all ten Rails/API2 cases. ENG-2683 adds the classic combined AI/Auditor Tickets workbook to API2 and moves the final one-off manual export action off Rails without changing the workbook or email flow.

• Sales-call compliance is globally enabled in US. The latest 24 hours produced 70 API2 result snapshots with 70 distinct run IDs, no fresh queue failure, and no waiting work. EU has no sales-call traffic or flag record. PR #10932  removes the regional selection branch so API2 owns every eligible sales call while non-sales-call post-sync analysis remains on Rails.

• ENG-2687  moves the six-hour sales-call credit reconciliation schedule from GoodJob to BullMQ. API2 preserves the 5,000-row bound, tenant checks, newest active evaluation-credit selection, idempotent usage insert, and atomic counter update. The Rails job class remains available for a code-revert rollback.

Current priorities

1. Reconcile existing owners, QA/tag/provider deployment observations and retention #11044. Finish or explicitly park waits before claiming a slot.
2. Remove the cold/expired-cache Rails session fetch, preserving existing cookies, tenant/role permissions, expiry and logout/revocation behavior. Continue login/logout, organization switching, CSRF/origin and identity work.
3. Replace authoritative Rails QA stage advancement and terminal-failure callbacks, including durable dispatch and retries. Keep result finalization and live billing explicit in the foundation sequence.
4. Resolve schema collision names and numeric/UUID mappings; prove compatible old/new deployments, upserts, FKs and agent-reader grants in local PostgreSQL. Production table moves follow the reviewed cutover plan.
5. Pull the next ready roadmap capability forward. Simulation porting is removed; its freed capacity goes to compliance gaps and schema preparation.

Historical rollout obligations — recheck current state

1. Verify deployment and advancing scans for merged Freshdesk repair #11026; prepare historical recovery and the EU coverage decision for specific approval. Observe ENG-2707 while independent QA, provider and HTTP packages progress.
2. Finish the deployed PII override revert’s clean window; successful regional callbacks are present and the matching Rails 422 remains absent.
3. Finish the residual transcription and Intercom guard observation windows; delete their orphaned controls only after both remain clean for 24 hours.
4. Verify deployment of the merged PR #10945 , then verify fresh successful Tickets exports in both regions with no failed or stuck export-runs before removing its Rails rollback path. Verify the deployed manual-export and user-requested QA revisions before deleting their controls.
5. Verify the deployed sales-call compliance bake-in with a 24-hour window with no duplicate result runs, terminal failure, or stuck analysis state.
6. Verify deployment of the merged PR #10946  for Zendesk stuck-conversation recovery, then verify four regional six-hour ticks before deleting the retained Rails job class.
7. Verify deployment of the merged PR #10949  for sales-call credit reconciliation, then verify its scheduler, usage rows, and counter deltas in both regions before crediting its production point.
8. Verify deployment of the merged automated-QA stack in ENG-2680 , repair stale rows per tenant, add the database invariant, then stage ticket-closed ownership.
9. Complete the remaining provider-sync callback domain, including the approved repair of the legacy Zendesk source with no subdomain; avoid treating individual flags or trigger branches as separate migrations.

How to use these notes

• Migration inventory records the current census, historical weights, state, parity evidence, fallbacks, and next action for every migration surface.
• GoodJob traffic records the comparable production baseline, retained queue snapshot, exclusions, and ordered retirement work.
• Rollouts and flags records temporary controls, regional state, bake conditions, and rollback order.
• Progress log records dated evidence, decisions, deployments, PRs, regressions, and ownership-gate changes.
• Zendesk cutover operations and conversation translation retain the detailed operational and implementation guidance for those workloads.

Completion criteria

The program is complete only when all of the following are true:

1. Every required webhook, background job, schedule, callback, integration, and synchronous app route is native to API2; there is no intentional Rails production dependency.
2. Production parity, idempotency, tenant isolation, retries, observability, and rollback behavior have been verified for each migrated workload.
3. First-party app traffic is cut fully to API2 after a measured dual-load parity window. The API2 public evaluation contract remains stable for Qonto and every other inventoried external consumer.
4. All required PostgreSQL objects live in public; no runtime or migration dependency references api2 or agent_api, and both application schemas are dropped through roll-forward migrations after their dependencies are removed.
5. GoodJob is empty and disabled, Rails web/worker infrastructure receives no traffic, and temporary flags, forwarding bridges, callback adapters, and rollback dependencies are removed after their bake windows.
6. US and EU remain healthy for at least 72 hours after the final ordinary cutover, including required actual events and weekly schedules; then the rulebase-api source project and its deployment infrastructure are deleted.