WorkOS authorization reconciliation

ENG-2815  transfers the daily 03:15 UTC WorkOS permission and role reconciliation from Rails’ recurring SyncWorkOSAuthorizationJob to API2’s maintenance BullMQ queue. Local role CRUD, WorkOS event processing, membership reconciliation and remote WorkOS writes remain separate ownership boundaries.

Contract

• Import only permission slugs in Rails’ generated assignable-permission set; ignore WorkOS widget permissions and unknown product permissions.
• Persist WorkOS permission IDs/slugs and current descriptions without changing the stable local key of an existing permission.
• Import environment roles globally and custom roles only for the matching local organization. Persist WorkOS IDs/slugs, names and descriptions.
• Derive collision-safe organization role keys exactly as Rails does. A remote organization role without a matching local organization fails closed.
• Add assignable remote permissions to a role without removing local permissions when WorkOS returns an empty or partial list.
• Serialize permission and role upserts with transaction advisory locks and preserve database uniqueness as the final concurrency guard.
• Isolate individual permission, environment-role, organization-list and organization-role failures. Report all counts and fail the aggregate BullMQ attempt so three bounded retries can recover transient errors.
• When a local organization references a WorkOS organization that does not exist in the current regional environment, skip that organization with a structured warning and report it separately from failures. This preserves Rails’ WorkOS::NotFoundError behavior; every other organization-list error remains retryable and fails the aggregate attempt.
• Do not log user identity, credentials or complete WorkOS payloads.

The Rails cron entry is removed in the same candidate release. Keep the Rails job and synchronization service for queued work, remote-write callers and code revert rollback.

Verification

Focused unit tests cover execution order, failure isolation, missing regional WorkOS organizations, environment-role deduplication and aggregate counts. PGlite tests exercise the production-shaped public tables and sequences, permission filtering, idempotent mappings, global and tenant-scoped role creation, additive permission behavior and orphan-role rejection. API2 typecheck, changed-file Rails lint and the GoodJob ownership assertion are required before push.

Rollout and rollback

Deploy the worker and Rails cron removal together. Confirm exactly one sync-workos-authorization-daily scheduler in each region and no new Rails recurring entry. Observe a real 03:15 UTC run per region with plausible counts, matching local/WorkOS mappings and permissions, no retained queue failure and no unresolved Sentry event. Require 24 hours after the actual run before recording production ownership.

Rollback removes the persisted BullMQ scheduler while the compatible API2 worker is still available, lets active work drain, then reverts the revision so the Rails cron resumes. Do not manually run both owners or repair authorization rows without separate production authorization.