Native existing-session validation

ENG-2730  removes the Rails HTTP session and organization-metadata requests from API2 authentication. Existing cookies continue to reference public.sessions; login, organization switching, WorkOS sign-out and session trimming remain separate ownership work.

Authentication and compatibility

API2 computes Rack’s SHA-256 private session key and reads the persisted session before every request, including identity-cache hits. Private storage keys are never accepted as bearer cookies. Legacy public-key rows remain readable and are secured on activity refresh. Missing/deleted/rotated sessions and changed Devise password salts return 401. Deleted accounts cannot authenticate; Rails accounts do not have a discarded_at column. The initial predicate referencing that nonexistent column requires the production repair in #11052. Discarded or foreign memberships do not confer organization access.

A bounded, data-only Marshal 4.8 codec reads the existing Base64 documents. It supports the session’s scalar, string, symbol, array, hash and reference forms; it never resolves Ruby classes or runs deserialization hooks. Invalid, unsupported, cyclic or oversized documents fail closed. Activity refresh uses compare-and-swap against the existing row and document, with one bounded re-authentication on conflict. It cannot recreate a session deleted by logout. Unrelated WorkOS, CSRF, flash and redirect state remains in the document.

The configured Rails retention policy is mirrored: 30 days from creation for the production cookie and 12 hours for the staging cookie. Staging also retains its 30-minute Warden idle timeout. Activity is refreshed at most once per minute; this does not extend the creation-based retention limit or issue a new cookie. Keep these constants aligned with rulebase-api/config/rulebase.yml until native session issuance and retention take ownership.

Account, organization, membership, role/permission and scope state is read natively. An authorization revision invalidates cached identity data when these change, including permission or membership-scope changes without a parent updated timestamp. Existing native directory renderers preserve account, membership, role, permission, employee and scope payloads. Tenantless signup keeps the existing 403 no_organization response.

Validation and rollout

• Focused codec, middleware, SQL and existing hydration tests cover expiry, revocation, rotation, tenant isolation, empty WorkOS password salts, scope and permission changes, conflicting refreshes and logout during refresh.
• Synthetic fixtures were generated by Ruby 3.3 Marshal/Base64. Ruby Marshal.load accepted 26 API2-written fixture documents, including integer boundaries and the Warden/WorkOS payload.
• A read-only September 3 01:18 UTC compatibility probe decoded the 1,000 most recently updated retained session documents in each region. All 2,000 decoded; seven US and two EU sampled documents had valid Warden authentication shapes. The sample is dominated by anonymous sessions and is not an exhaustive active customer census. All session primary, session-ID and updated-at indexes were valid. No session payload, cookie, token or account identity was returned from production, and no production session was updated by the probe.

Before merge, require local checks, CI and required review. After automatic production deployment, verify cold and warm authentication in US/EU, invalid cookie denial, real sign-out/rotation behavior and no regression in session 401/403/500 rates. Observe database query latency and pool pressure: warm requests now validate persisted authorization instead of trusting a five-minute cache. Record observations at 30, 60 and 120 minutes and through 24 hours, with actual signed-in activity. Do not infer success from a deployment or empty error window.

A code revert restores Rails HTTP validation. API2-written documents remain Ruby Marshal-compatible; the Rails session/identity tables, login routes and trimming job remain present. No schema migration, rollout flag or production data repair is part of this package. Full native login/logout, WorkOS lifecycle and removal of Rails session storage/scheduling remain explicit follow-ups.