Rulebase 2.0 Architecture Plan

Date: 2026-04-15 (originally drafted); last reality-check: 2026-07-22 Status: Largely shipped and evolved past the original plan — see Current state (2026-07) below. Sections below the status map keep the original design narrative; treat marked historical blocks as archive, not active roadmap. Scope: Rebuild the runtime behind Rulebase 2.0’s primitives as a TypeScript service (api2), running alongside the existing Rails API during a strangler migration.

Current state (2026-07)

This document mixes original design intent with what shipped. Prefer code over prose when they disagree: rulebase-web/rulebase-api2/src/db/schema.ts, src/lib/agent-tool.ts, src/lib/agent-trigger.ts, src/lib/agent-graph.ts, and src/lib/rails-client.ts.

Shipped and in production:

• api2 Hono service on Fargate (api2.rulebase.co / eu.api2.rulebase.co), web + BullMQ worker. ~227 Drizzle migrations; large native surface beyond the original handful of tables.
• Workflows (managed agents) live in workflows with instructions, tools jsonb, triggers jsonb, optional graph JSONB, version history (agent_versions), and node_executions. Flat instruction mode and graph/workflow execution mode both exist (executionMode: legacy_instruction | workflow).
• Onboarding agents are a separate table (role-scoped); do not confuse them with managed-agent workflows.
• Runner: BullMQ + Redis + Vercel AI SDK (not Inngest, not Mastra). Bull-Board at /queues.
• Tools: rulebase, slack, email, microsoft_teams, snowflake, integration, mcp_server, openapi (see agent-tool.ts). Mix of Rails proxies and TS-native clients (e.g. Front, Intercom, Linq under src/lib/).
• Triggers: schedule (hourly/daily/weekly/monthly), ticket created/updated/closed/solved, sales call ended, Jira, Linear, email, Slack, webhook. Rails → api2 webhook forward path exists (ForwardRulebaseWebhookJob → POST /webhooks/rulebase).
• Chat / Instruct / artifacts, coaching, custom dashboards, work items, knowledge-base documents, memory, credits (credit_usage_events + billing/spend-cap tables).
• query_data on api2 (validator + executor in-process; see query_data to api2 plan).
• MCP server at mcp.rulebase.co / eu.mcp.rulebase.co (customer-facing read tools; see external MCP guide ).
• TipTap Pages product surface removed from the app (legacy /pages URLs should not be linked). Compliance/testing sheets remain primarily Rails; onboarding has api2 customers_sheets and related sheet demos.

Still open / deferred (relative to the original plan):

• Full Sheets/Pages port to api2 as described in the sections below (Pages port is historical; Sheets port only partially realized).
• Inngest self-host (still deferred while BullMQ is enough).
• Mastra (still not adopted).
• Generic GET /internal/organizations/:id/integration_credentials/:type — many tools still use per-integration patterns / Rails proxies.
• Uniform TS-native execution for every vendor tool (direction unchanged; progress is uneven by integration).
• End-to-end X-Rulebase-Trace-Id propagation.

Historical checkpoints from May 2026 are folded into Progress. The May sync-tier learnings section at the bottom remains current guidance for any future sync port.

Context

The 2.0 product reframe is Review engine → Actions engine. Instead of shipping fixed surfaces (Compliance Testing, Marketing Review, Conversation QA, Disputes, etc.), we collapse to a small set of composable primitives: Workflows, Agents, Pages, Sheets, Rubrics. Positioning shorthand: Stack AI / Gumloop, but for financial services — general-purpose agent/workflow builder, verticalized with FS-native tool integrations (Unit21, Alloy, Galileo, Zendesk, Jira, Salesforce, Looker) and compliance context built into the primitives.

Pricing moves from fixed annual (based on volume estimates) to consumption-driven. Enterprise customers have mandates to increase token spend to drive AI adoption; the product has to scale with their use, not cap it.

What already exists

Minimal versions of the primitives are already in Rails + rulebase-ui:

• Workflows — full visual builder with graph canvas, node edit forms, triggers (schedule + Jira/email/Slack/ticket events), variables, entities, run history, Sentinel (“chat with your workflow”) AI builder. ~5,700 LOC across Workflow* models and WorkflowNode::Run::* runtimes. 17 agent-node tools and 13 outer-run tools.
• Pages — TipTap editor with custom PageLinkNode, slash command extension (nested pages, sheets, AI ask, AI write), TOC, markdown export, debounced HTML save. AI chat side panel wired to sentinel_conversations.
• Sheets — AG Grid Enterprise, typed columns (string/enum/date/boolean/url/record/group) via StoreModel discriminated unions, realtime via SheetChannel (6-event protocol), version history with preview_at_version + transactional restore, reasoning citations (polymorphic citable). Fill runtime is already a tool-calling LLM loop in Sheet#fill_with_llm.

None of these are truly launched. Some basic use cases run; no customer has meaningfully adopted. This gives us unusual freedom to pick clean boundaries over safe ones during the rewrite — no behavioral parity needed, no data migration risk.

Why a new service, not more Rails

The Rails workflow runtime works but has three structural limits for 2.0:

1. Durable execution gap. GoodJob retries jobs; it does not checkpoint a 20-step agent run and resume from step 14 after a Fargate deploy. For workflows that run minutes to hours across multiple tool calls (disputes, marketing reviews, compliance testing), this is a trust problem the moment it bites a customer.
2. LLM ecosystem inertia. Vercel AI SDK, Mastra, LangGraph, MCP host implementations, Composio/Arcade, AI SDK UI — the center of gravity is TypeScript. rubyllm is closing some of the gap but the DX for agent-shaped work is measurably behind.
3. End-to-end types for demo velocity. The north-star UX is “build a workflow live on a demo call” via the Sentinel. The Sentinel emits a workflow graph that the React canvas renders immediately; agents emit cell values, citations, reasoning, tool arguments that the UI consumes. Shared Zod schemas between agent output and UI collapse a class of glue code that matters for iteration speed.

Stack decisions

Non-goals

• Porting Zendesk/Aircall/Salesforce/Intercom sync to api2 in v1. These stay in Rails for the duration of the strangler; api2 reads Rails-owned data via internal endpoints. Eventually-migrate, not indefinitely-Rails — see Default ownership rule below. When the sync-port does happen, it has specific primitives requirements established by a 2026-05 production incident — see Sync-tier learnings below.
• Rewriting Rails domain models (conversations, disputes, compliance_cases) in v1. api2 accesses them via Rails internal HTTP endpoints. Same eventually-migrate caveat.
• Replacing ActionCable before phase 2.
• Building an external customer API in 2.0 v1. Wait for real demand, then design as a v1 product.

Long-term direction

The end state is all of Rulebase on Node/Hono. Rails goes away. The strangler in this plan is the first chapter, not the whole book. The “Rails keeps owning” list below is what stays on Rails for the duration of v1 — every entry on that list is a future migration project, sized and scheduled separately. None of it is permanently Rails.

This matters for how new features get scoped: see the rule below.

Architecture

Topology

app.rulebase.co        → React SPA (CloudFront)
api.rulebase.co        → Rails
api2.rulebase.co       → Hono + BullMQ worker (Vercel AI SDK)
mcp.rulebase.co        → MCP (api2 /mcp)
admin.internal…        → Internal admin UI (rulebase-admin)
eu.api.rulebase.co     → Rails EU
eu.api2.rulebase.co    → Hono EU
eu.mcp.rulebase.co     → MCP EU
eu.admin.internal…     → Admin EU

Historical note: Early drafts showed Mastra + self-hosted Inngest here. Neither shipped; BullMQ + Vercel AI SDK remain the runtime. Inngest may still return if durable mid-run resumption earns the cost.

The UI holds two typed API clients side by side: existing rulebaseApi (from Rails OpenAPI) and new api2 (from Hono-generated OpenAPI). Each feature calls whichever service owns its data.

Data layer

One RDS cluster, schemas in use today:

• public — Rails/ActiveRecord owns. Unchanged.
• api2 — Drizzle owns. New 2.0 tables.

Historical note: An inngest schema was planned for self-hosted Inngest durable state. Not created while Inngest remains deferred.

Drizzle config: schemaFilter: ["api2"]. Rails schema_search_path: "public". Per-service Postgres roles enforce the boundary:

rails_app_user    → ALL on public
api2_app_user     → ALL on api2

No cross-schema FKs or joins. api2 is fully self-contained. It maintains its own api2.organizations table (synced from Rails via slug on first request) and all FKs within api2 reference api2 tables. When api2 needs Rails-owned data (conversations, disputes, credentials), it calls Rails internal endpoints — never reads from public.* directly. This means the api2 Postgres role needs zero privileges on the public schema.

Primary keys: UUIDv7 everywhere in api2. Time-sortable (cursor pagination without a separate created_at index), single column, no prefix-id mapping layer. Generated application-side via the uuidv7 package. Rails keeps its bigint PKs; the boundary is the internal HTTP API, not shared FKs.

Same cluster means backups, PITR, failover, residency all remain one concern across three regions.

Connection pooling: both services behind RDS Proxy or PgBouncer (transaction mode). With current Fargate instance counts (Rails API + workers) plus api2 web + worker on top, this matters before we think it will.

Service boundary

Default ownership rule

New features ship on api2 by default. They fall back to Rails only when they fundamentally require a Rails-owned primitive — RLS-enforced SQL execution, integration sync, encrypted credential storage. Where a feature needs Rails-owned data, the Rails side exposes a thin internal HTTP endpoint; the feature’s logic lives in api2.

This swaps the implicit default of the original plan (“which side does this fit best?”) for an explicit one (“api2 unless there’s a hard reason”). It’s how we avoid quietly accreting more surface on Rails over the strangler window — every new Rails-side write extends the eventual migration, so we add to Rails only when there’s no other choice.

Concretely:

• New tables → api2 schema, Drizzle. Not new columns on Rails models, even when the data is “about” a Rails-owned entity. Use foreign-key-by-id (rails_conversation_id text on the api2 row) instead of jsonb columns added to public.conversations.
• New OAuth-style tools → TS-native where practical, fetching credentials on demand. Prefer not to add new Rails proxy endpoints for brand-new tools.
• New scheduled / event-driven workflows → BullMQ in api2 (Inngest if/when adopted). Not new GoodJob jobs in Rails for agent work.
• New UI surfaces wired to new data → call api2 directly. Not Rails endpoints that join api2 data with Rails data.

The handful of things that genuinely require Rails primitives (decrypting some credentials, Devise / WorkOS / Stripe, much of compliance reporting and sync) stay on Rails — but those are the exceptions, not the default. Agent query_data SQL now runs in api2 against q_* views; see the query_data migration plan.

Rails keeps owning (for the duration of v1)

• Domain data: users, organizations, accounts, conversations, disputes, compliance_cases, audit events, etc.
• Integration sync: Zendesk, Aircall, Salesforce, and other webhook/polling sync that has not moved (Front/Intercom/Linq have growing api2-native surfaces).
• Compliance test scaffolding, marketing review pipelines, much of the QA evaluation runner stack (api2 owns pieces; Rails still owns a large share).
• Integration credential storage (ActiveRecord encrypted attributes) for Rails-owned integrations.
• WorkOS SSO, Devise JWT minting, Stripe billing, compliance reporting.

Each of these is a future migration project, not a permanent residence.

api2 owns

Synced mirrors of Rails entities (shipped): lazy-upsert-on-first-touch from Rails. All carry a rulebase_id text unique link back to the Rails source-of-truth row.

• users, organizations, memberships — identity surface.
• integrations, connections — integration/data-source registry. connections link an org to an integration with the corresponding Rails connection_id.
• conversations — mirrored conversation rows (id + rulebase_id + org + connection). Used as the api2-side anchor for conversation-related state (see Rho escalations agents; alert dedup later moved to Rails agent_alert_tags).

Native api2 entities (shipped; non-exhaustive):

• workflows — managed agents: instructions, tools, triggers, optional graph, executionMode, schedule indexes. Versioned via agent_versions. Runs in agent_runs (+ node_executions when graph mode runs).
• agents — onboarding-role agents (separate from managed-agent workflows).
• Chat / Instruct: chats, chat_messages_v2, artifacts, artifact_versions.
• Credits / billing: credit_usage_events, organization_billing_settings, membership_spend_caps.
• Broader product surface also lives here: coaching, dashboards, work items, knowledge-base documents, memory, tasks/onboarding cases, SLA outcomes, and more. See schema.ts.

Native api2 entities (planned in the original doc; status as of 2026-07):

• TipTap Pages / PageRevisions port — historical; product Pages routes removed. See Pages port.
• Full Sheets port (columns/rows/cells/versions) — partial; Rails sheets remain primary for compliance testing; api2 has customers_sheets, agent_run_sheets, and onboarding sheet demos. See Sheets port.

Execution + transport:

• Agent execution loop on Vercel AI SDK inside BullMQ workers (separate worker.ts entrypoint).
• Streaming endpoints (SSE for agent / chat output today; WS for Yjs collab was phase-2 speculation and is not active work).

Rails ↔ api2 bridge

User auth: Rails mints a Devise JWT on login (existing flow, unchanged). Both services validate with DEVISE_JWT_SECRET_KEY from SSM. api2 has a Hono middleware that reads Authorization: Bearer, verifies, sets c.var.accountId + c.var.orgId (latter resolved once per session and cached; invalidated on membership change webhook from Rails).

Service-to-service: separate short-lived HS256 JWT with INTERNAL_SERVICE_SECRET. Claims: {iss, org_id, run_id, exp}. Used in both directions.

Rails internal endpoints that api2 calls today (see src/lib/rails-client.ts; all authenticated with RAILS_INTERNAL_API_KEY bearer):

• POST /internal/tools/call — generic tool dispatcher used for query_data, ask_knowledge_base, create_jira_issue_suggestion, bulk_assign_conversations, qa_evaluation_coverage, top_recurring_issues, qa_score_trend, top_quality_gaps, agent_performance_summary, reopen_rate, knowledge_base_gaps. This is the actual current pattern, replacing the per-endpoint /internal/tools/<name> shape originally drafted.
• POST /internal/slack_notifications — sends a Slack message via the Rails-side OAuth credentials. Takes { organization_id, organization_data_source_id, channel_id, text }. Returns { ok, ts?, error? }.
• GET /internal/organizations/:id — basic org metadata (id, name, timezone).
• GET /internal/evaluations/:id/context and POST /internal/evaluations/:id/results — used by the conversation QA evaluation runner.
• GET /session and GET /organization_data_sources — cookie-authenticated, used by the Hono session middleware to resolve account + org for incoming UI requests.

Not yet built / partial (originally drafted):

• GET /internal/organizations/:id/integration_credentials/:type — generic credential fetch for TS-native tool execution. Some tools still use per-tool Rails proxies; others have api2-native clients. Build or skip per integration.
• POST /internal/broadcast/sheet/:id / /internal/broadcast/workflow/:id — full Sheets/Workflows ActionCable bridge from the original plan; not the active path for managed agents today.
• POST /internal/workflow-runs/:id/events — superseded by Rails ForwardRulebaseWebhookJob → api2 POST /webhooks/rulebase for event triggers.

TS-native tools (direction; uneven progress): Slack, Zendesk, Jira, Salesforce, Google Sheets, Looker, Intercom, Freshdesk, Front, Linq, webhooks, MCP servers. The default ownership rule still prefers TS-native for new work. As of 2026-07, several integrations have api2-native clients under src/lib/<provider>/, while many tools still execute via Rails proxies. The generic credential-fetch endpoint below remains optional infrastructure, not a blocker for every new tool.

Trace propagation (X-Rulebase-Trace-Id) is in scope for all of these but not yet plumbed end-to-end.

Trace propagation

Every request generates or forwards X-Rulebase-Trace-Id. The ID appears in:

• api2 logs
• Rails logs (via middleware)
• Inngest run metadata (set on event send)
• OpenAI request metadata field (per Helicone convention — existing practice)
• ActionCable broadcast payloads

Cheap on day one, invaluable during production incidents.

Module layout inside api2

Reality (as of 2026-05): The codebase has converged on a flatter db/ + lib/ + routes/ + prompts/ structure rather than the modules/<domain>/{schema,service,routes,policy,tools,inngest}.ts layout originally sketched. Drizzle tables live together in src/db/schema.ts. Domain logic lives in src/lib/<feature>.ts files (e.g. agent-tool.ts, agent-trigger.ts, rails-client.ts, chat-runtime.ts, conversation-qa-evaluation-runner.ts). The block below is the original aspirational layout — kept here as a reference for if/when domain growth justifies a per-module split.

rulebase-api2/src/
  db/
    client.ts                     # Drizzle instance
    schema.ts                     # Re-exports
  modules/
    organizations/
      schema.ts                   # api2.organizations (synced mirror)
      service.ts                  # Upsert-on-first-request logic
    agents/
      schema.ts                   # agents table (graph JSONB)
      types.ts                    # Zod schemas (agentGraphSchema, node configs)
      service.ts                  # Business logic (fat services)
      routes.ts                   # Hono routes (thin)
      policy.ts                   # Authorization functions
      tools.ts                    # Runtime tool definitions
      inngest.ts                  # Durable execution functions
    triggers/
      schema.ts                   # agent_triggers table
      types.ts                    # Trigger config Zod unions
      service.ts
      routes.ts
      inngest.ts                  # Schedule + event trigger functions
    runs/
      schema.ts                   # agent_runs
      service.ts
      routes.ts
    sheets/
      ... (same module shape)
    pages/
  lib/
    auth.ts                       # JWT middleware
    rails.ts                      # Internal Rails client
    errors.ts                     # Typed errors + HTTP mapper
    trace.ts                      # X-Rulebase-Trace-Id propagation
  inngest/
    client.ts
  mastra/
    index.ts                      # Mastra setup, tool registry
  server.ts                       # Hono app wiring

Discipline

• Fat services, thin routes, dumb schemas. Drizzle tables are declarations; no callbacks. Business logic lives in service functions.
• Explicit tenancy. Every service function takes ctx: { orgId, accountId, tx? }. No ambient ActsAsTenant magic.
• Services accept optional transactions. If ctx.tx provided, use it; otherwise open db.transaction.
• Authorization in routes, not services. Services stay reusable from Inngest + Mastra without rechecking.
• Typed errors throw, Hono error handler maps to HTTP. Like Rails rescue_from.
• Zod schemas carry .openapi("Name") for clean generated spec.
• Presenter functions convert Drizzle rows to output types — don’t return rows directly from routes.
• Data model is the source of truth, Mastra is the executor. Agents are Drizzle rows with a graph JSONB column; Mastra workflows are compiled from the graph at runtime. If Mastra pivots, swap is mechanical.
• Tools defined in a framework-agnostic shape: { name, description, input: z.object, handler }. Adapters register them with Mastra; same definitions would work with LangGraph TS.
• UUIDv7 for all api2 primary keys. Single column, time-sortable, no prefix-id mapping layer. Generated application-side via uuidv7 package.
• No cross-schema reads. api2 maintains its own organizations table (synced from Rails via slug). All FKs within api2 reference api2 tables. Rails data accessed only via internal HTTP endpoints.

Agent model (replaces Workflow model)

Reality (as of 2026-07): Managed agents live in the workflows table. They support both flat instruction-driven runs (executionMode: legacy_instruction) and graph documents (executionMode: workflow with graph JSONB, versioned in agent_versions, with node_executions for graph steps). Tool and trigger configs remain typed arrays on the workflow row (agent-tool.ts, agent-trigger.ts). A separate agents table is for onboarding-role agents, not the managed-agent builder.

Earlier (2026-05) we shipped the flat instruction model first and deferred the graph document. The graph path has since landed; the original “document-shaped” design below is closer to today’s workflow mode than it was in May, but node-type inventory and Mastra compilation details remain historical — execution is still Vercel AI SDK inside BullMQ, not Mastra.

Triggers remain inline on workflows.triggers (no separate agent_triggers table). Per-step LLM transcripts still primarily live in Braintrust via wrapAISDK; node_executions tracks graph-node runs rather than every model/tool turn. See Agent run observability.

Document-shaped, not relational (workflow mode today; prose below is the original design sketch)

In Rails, a workflow is spread across 6+ tables: workflows, workflow_nodes, workflow_edges, workflow_triggers, workflow_entities, workflow_variables. This normalization exists because of Rails conventions (has_many, accepts_nested_attributes_for), not because there’s a data modeling reason for it. You never query “find all if-nodes across all agents” — you always load the full graph at once.

In api2, the graph is a document. One agents row, one graph JSONB column:

const api2 = pgSchema("api2");
 
export const agents = api2.table("agents", {
  id: uuid("id").$defaultFn(() => uuidv7()).primaryKey(),
  organizationId: uuid("organization_id").notNull().references(() => organizations.id),
  name: text("name").notNull(),
  instructions: text("instructions"),
  graph: jsonb("graph").notNull().$type<AgentGraph>(),
  createdAt: timestamp("created_at").defaultNow().notNull(),
  updatedAt: timestamp("updated_at").defaultNow().notNull(),
});

Zod validates the graph shape:

const agentNodeConfigSchema = z.discriminatedUnion("type", [
  z.object({ type: z.literal("start") }),
  z.object({ type: z.literal("agent"), name: z.string(), instructions: z.string(), tools: z.array(toolConfigSchema) }),
  z.object({ type: z.literal("if"), conditions: z.array(conditionSchema) }),
]);
 
const agentGraphSchema = z.object({
  nodes: z.array(z.object({
    id: z.string(),
    type: agentNodeConfigSchema,
    position: z.object({ x: z.number(), y: z.number() }),
  })),
  edges: z.array(z.object({
    id: z.string(),
    source: z.string(),
    target: z.string(),
    sourceHandle: z.string().optional(),
  })),
});

This means:

• Save = one PUT. UI sends the full React Flow graph JSON; api2 validates with Zod, writes one row. No nested attributes, no diffing, no orphan cleanup.
• Mastra compiles from the JSONB directly. Read the graph column, walk nodes/edges, build the Mastra workflow at runtime. The document IS the source of truth.
• Version history = snapshots. Store previous graph values. One blob per version, not a join across five tables.

What gets its own table (independent lifecycle)

Not everything collapses into the graph JSONB:

• agent_triggers — queried independently (“find all schedule triggers due now”), have their own execution cadence. Own table.
• agent_runs — execution records, queried by status/time/org, never edited after creation. Own table. (Originally drafted as agent_runs + agent_run_steps; the per-step table was reversed — see Agent run observability.)
• agent_variables — may be shared across agents or referenced from triggers. Own table (or JSONB on the agent row if they stay agent-scoped — decide during spike).

Node types: clean slate

Rails has 12 node types, most of which are deprecated specialized actions (create_zendesk_ticket, append_to_google_sheet, etc.) that the TODO says should be replaced by agent nodes with tools. In api2, start clean:

• start — entry point, no config.
• agent — LLM + instructions + tools. This is the workhorse. What was previously create_jira_issue becomes an agent node with a create_jira_issue tool.
• if — conditional branching with CEL expressions (port via cel-js).

Three node types. Everything else is a tool attached to an agent node.

CellValue: don’t port

CellValue (template | static | ref | fn | value) was a mini-language for passing data between specialized nodes. With only agent nodes + if nodes, data flows through the LLM context — agents resolve references naturally. CellValue dies with the specialized nodes. No port needed.

Triggers → Inngest events (historical target; today: BullMQ + Croner + webhooks)

Reality (as of 2026-07): Schedule triggers fire via Croner inside the BullMQ worker (workflows.next_run_at / schedule trigger state drives enqueue). Event triggers are typed in lib/agent-trigger.ts; Rails forwards matching events into api2 via ForwardRulebaseWebhookJob → POST /webhooks/rulebase. Inngest event filters below remain the optional future shape if we adopt Inngest.

WorkflowTrigger.matches_event? becomes Inngest event filters:

inngest.createFunction(
  { id: "agent-jira-issue-created" },
  { event: "rails/jira.issue_created", if: "event.data.request_type_id == async.data.request_type_id" },
  async ({ event, step }) => {
    // load agent, compile graph, execute
  }
);

schedule triggers become Inngest cron functions. Rails webhooks (Jira, Zendesk, email inbound, Slack) forward into api2’s Inngest event bus via inngest.send(...) from a Rails controller.

AgentRun → Inngest run (historical; today: BullMQ job → agent_runs row)

Reality (as of 2026-07): Each run is a BullMQ job that writes to agent_runs (and node_executions when running graph mode). See Agent run observability.

agent_runs retained in api2 for UI/history. If we adopt Inngest later, run-state transitions could move from the worker’s try/catch to Inngest event handlers; the row shape does not need to change for that.

Agent run observability

Two questions get conflated when designing run persistence: “what configuration produced this run?” and “what did the model do step-by-step?” They have different answers.

What the row carries (today):

agent_runs {
  // identity / lifecycle
  id, agent_id, organization_id, status, source,
  scheduled_for, bullmq_job_id, started_at, finished_at,
  // configuration snapshot
  system_prompt text,           // composed prompt as it was sent to the LLM
  // outcome
  output text,                  // final assistant text
  error text,                   // error message on failure
  total_input_tokens int,
  total_output_tokens int,
  total_reasoning_tokens int,
}

system_prompt and the token totals are useful regardless of any eval harness: prompt for “what config did this run actually see” (since agents.instructions and agents.tools can drift over time); tokens for cost tracking.

Why no agent_run_steps table. The originally-planned per-step table would store one row per LLM call / tool call with the request/response payloads. We don’t ship it because Braintrust is already that store. The agent loop runs through wrapAISDK(ai) (see lib/braintrust.ts), so every model call, tool call, tool result, and usage delta is automatically traced to Braintrust. Persisting the same data in Postgres would be:

• Bloated. Single rows could be MBs of request.body / response.body / per-step messages.
• Coupled to AI SDK shape. SDK version bumps would propagate to every reader (admin UI, analytics).
• Redundant. Anything you’d query Postgres for (“what tools did this run call”) you can answer in Braintrust today.

We’d add agent_run_steps (or a denormalized agent_run_tool_calls sibling table) the day we have a concrete consumer Braintrust can’t serve — most likely a customer-facing run viewer, or a SQL analytics query against tool calls. Until then, the output + system_prompt + token totals are enough for our own admin debugging.

Sentinel / Instruct port

Reality (as of 2026-07): In-product chat / Instruct is live on api2 (chats, artifacts, tool catalog, app navigation skills). The original “Sentinel mutates a workflow graph via Mastra tools” design below is historical; graph editing today is the managed-agent UI + versioning, not a Mastra Agent tool loop.

The Sentinel (“chat with your agent”) was the crown-jewel demo feature in the original plan. Port-as-Mastra sketch: tools operating on the graph JSONB directly (GetAgent, UpdateGraph, AddNode, …). Kept here as archive.

Consolidated tool registry

Rails has two tool systems (WorkflowNode::Run::Agent::Tools::* and WorkflowRun::Tools::*). Collapse into one in api2. Every tool defined once; multiple contexts consume the same definition. The Sentinel’s agent-editing tools and an agent-node’s runtime tools share the same registry pattern.

Sheets port

Status (2026-07): Historical target for a full TipTap/AG Grid sheets rewrite on api2. Compliance testing sheets remain Rails-owned. api2 has related pieces (customers_sheets, agent_run_sheets, onboarding sheet demos) but not the full column/row/cell/version port described below. Keep this section as design archive.

Shape port 1:1. Column types map cleanly to a Zod discriminated union:

export const SheetColumn = z.discriminatedUnion("column_type", [
  z.object({ column_type: z.literal("string"),  options: z.object({ type: z.literal("string") }) }),
  z.object({ column_type: z.literal("enum"),    options: EnumOptions }),
  z.object({ column_type: z.literal("date"),    options: z.object({ type: z.literal("date") }) }),
  z.object({ column_type: z.literal("boolean"), options: z.object({ type: z.literal("boolean") }) }),
  z.object({ column_type: z.literal("url"),     options: z.object({ type: z.literal("url") }) }),
  z.object({ column_type: z.literal("record"),  options: RecordOptions }),
  z.object({ column_type: z.literal("group"),   options: z.object({ type: z.literal("group") }) }),
]);

The record-type column is the key cross-boundary concern: cells store {record_type, entity_id}; rendering hydrates the display by calling Rails. Cache hydration results aggressively; the grid paginates, so N+1 risk is bounded.

Sheet#fill_with_llm ports to an Inngest workflow: one step.run("build-prompt"), parallel step.run for tool calls, one step.run("update-cell") per cell. Each step is checkpointed. A Fargate task restart during a fill resumes mid-run.

AG Grid stays. The six-event SheetRealtimeMessage contract stays. Only the transport and data source change under the hood.

Pages port

Status (2026-07): Historical. Product Pages routes (/pages, /pages/{pageId}) were removed. Do not treat this section as active roadmap. Instruct / artifacts cover much of the “agent-native document” need differently.

Keep TipTap. Migrate storage from HTML string (ed.getHTML()) to ProseMirror JSON (ed.getJSON()) in the api2 port — structured storage enables agent-native blocks (streaming tokens, sheet-row refs, citations) and preserves attributes properly.

Agent-native block types to add as TipTap custom nodes (same pattern as existing PageLinkNode):

• AgentRunNode — streams tokens, shows inline tool calls, expandable reasoning, status indicator.
• SheetRowRefNode — live-updating reference to a sheet row.
• CitationNode — structured citation with polymorphic citable (conversation_part, transcript_message, framework_document, program_policy). Extends existing reasoning citation shape.

Collaboration via Yjs + @tiptap/extension-collaboration + Hocuspocus server lands in phase 2 when the first customer needs human + agent co-editing. Don’t preempt.

Deployment

Flightcontrol services added per region (dev, prod US, prod EU):

• rulebase-api2-{env} — Hono web server, Fargate. Entrypoint: dist/server.js.
• rulebase-api2-worker-{env} — BullMQ worker, Fargate. Separate entrypoint dist/worker.js (own scaling profile from the web tier). (Originally drafted as rulebase-inngest-{env} for an Inngest self-host; that service is deferred — see Stack decisions.)
• rulebase-hocuspocus-{env} — deferred to phase 2.

api2.rulebase.co / eu.api2.rulebase.co subdomains via CloudFront. SSE support requires disabling response buffering on that path. Bull-Board UI mounted at /queues for ops visibility into the BullMQ worker.

Environment variables from Parameter Store under /rulebase/{env}/:

• DATABASE_URL (api2 role, scoped schemas)
• REDIS_URL (shared by web + worker for BullMQ)
• DEVISE_JWT_SECRET_KEY (shared with Rails)
• RAILS_INTERNAL_BASE_URL, RAILS_INTERNAL_API_KEY
• INNGEST_EVENT_KEY, INNGEST_SIGNING_KEY — not yet needed; reintroduce when Inngest is adopted.
• Per-integration OAuth client IDs/secrets (shared with Rails; same SSM paths)
• OpenAI / Anthropic / Braintrust keys (shared with Rails)

No new vendor keys. No new subprocessor disclosure.

Validation milestones (order of execution)

Historical (2026-04 plan). These were the original dogfood milestones. Several product paths shipped under different names (Instruct, managed agents, onboarding, credits, MCP, dashboards). Keep for chronology; do not treat as the current backlog.

Selected to prove primitives correctness with minimum external-dependency risk.

1. Operational readiness. api2 + Inngest on Fargate dev. /health endpoints. JWT bridge to Rails working both ways. Drizzle migrations applied. Tri-region deploy-target confirmed.
2. Sara’s weekly ticket audit report. Schedule trigger → query Salesforce (TS-native tool) + Zendesk (TS-native tool) → write rows into an api2 sheet → post to Slack. Cleanest primitives test, minimum Rails surface touched. First real workflow.
3. Rho’s disputes workflow. Jira ticket created (Rails webhook → api2 Inngest event) → row in sheet. Ticket status change event → update row. Schedule poll Galileo Looker Studio until dispute resolved → write transaction details. Proves step.waitForEvent + long-running durable execution.
4. Dispute Jira ticket suggestions. Replaces existing suggest_jira_issue_drafts. Exercises agent-node tool registry end to end.
5. Rho’s Unit21 reporting (pages). Agent writes a structured report into a Page with embedded sheet references. Proves the Page + AgentRunNode + SheetRowRefNode surface.
6. Custom dashboards with embedded charts (pages). Recharts in custom TipTap nodes.
7. Marketing reviews as email-triggered workflows. Rails inbound email → api2 event → marketing review agent → Page output.
8. Kuda’s ticket assignment. Weekly schedule → query scores → round-robin assign to N people via Rails internal endpoint.

Each milestone ships as an internal dogfood before a customer sees it. Sentinel must be able to build each workflow via chat by milestone 3 — otherwise we’ve lost the demo-velocity thesis.

Risks

Mastra framework bet. 1.0 was January 2026; framework could stall, pivot, or get acquired. Mitigation: data model primary, tools framework-agnostic; swap to LangGraph TS is mechanical if needed.

Dual-maintenance window. 9–12 months of Rails workflows and api2 workflows coexisting. Customers on the old engine can’t use 2.0 features. Keep the migration per-customer list concrete; don’t let the old engine become a zombie that accumulates patches forever.

CEL behavioral drift. cel-js might diverge from the Ruby cel gem on edge cases in if-node conditions. Cover with a porting test suite.

EU residency from day one. api2 must deploy to eu-central-1 before any EU customer can run a 2.0 workflow. Stand up the EU service in week 1, not week 30.

Sentinel behavioral parity. The LLM-builds-workflow surface is the single most visible piece of the migration and the most demo-sensitive. Budget more time than feels comfortable.

Record-type cross-service hydration. Every render of a sheet with record columns hits Rails. Cache aggressively, paginate well, monitor latency.

No net-new subprocessor. Mastra, Hono, Drizzle, Zod, Inngest (self-hosted) are libraries or self-hosted services. No customer disclosure required. Any addition (e.g., if we ever consider Inngest Cloud, Temporal Cloud, a managed vector DB) triggers the SOC 2 subprocessor flow — don’t slip one in casually.

Not doing

Several “not doing” items from the original plan have since shipped or changed. Strikethroughs below mark those; the rest still hold.

• No external customer-facing API in 2.0 v1. Partial public API v2 surface exists (work items, evaluations, conversation uploads); still not a full external platform API.
• No wholesale Zendesk/Salesforce/Aircall sync port to api2. Rails keeps most sync; individual providers may grow api2-native clients.
• No replacement of ActionCable as a global realtime bus in phase 1.
• No Yjs/Hocuspocus collab as an active project.
• No BlockNote migration.
• No rewrite of rulebase-ml. Stays on Modal, scoped to transcription/translation/redaction/KB chunking.
• No MCP server in v1. Shipped — see customer MCP guide  and api2 routes/mcp.ts.

Open questions

1. KB / compliance policies — are these also Pages? Likely yes, but need a concrete data-model decision before milestone 5.
2. Marketing review — app on top of primitives, or curated surface? Lean toward curated app using the primitives underneath (better for enterprise buyers who want reproducible outputs).
3. MCP — host only, consumer only, or both? Suggest host first (expose api2 as MCP server so Claude/other clients can call tools), consumer second (agent nodes can call user-provided MCP servers).
4. Agent performance / coaching module port. Deferred; revisit after milestones 1–3.
5. Consumption pricing meter shape. AgentRun + UsageEvent tables need a concrete schema before milestone 2 (metering must be in place from the first real production workflow, not retrofitted).

Go/no-go checkpoint

Historical. The one-week spike below already happened; api2 is in production. Kept for archive.

One-week spike before committing to the full plan:

• Drizzle schema for organizations, agents (with graph JSONB), agent_triggers, agent_runs in api2 schema. UUIDv7 PKs.
• Zod schemas for graph nodes (start, agent, if), edges, trigger configs.
• Hono server on Fargate dev with @hono/zod-openapi, JWT middleware, one route that returns an agent with its graph.
• Mastra compiling a graph from the JSONB and executing a simple agent-node + if-node workflow via Inngest.
• UI can call api2.GET("/agents/{id}") via openapi-fetch, render the graph in React Flow from the returned JSONB.

If the code feels cleaner than the Rails equivalent — better types, faster iteration, clearer boundaries — commit. If it feels like a lateral move, reconsider.

Don’t commit on vibes. Commit after one concrete spike tells you whether the new substrate actually lets you move faster.

Progress

Last updated: 2026-07-22

Shipped (foundations through mid-2026)

Foundations (originally listed under “Completed” 2026-04-16):

• Hono project scaffolded inside rulebase-web/rulebase-api2 (pnpm workspace, TS strict, ESM, tsx watch).
• OpenAPI wiring via @hono/zod-openapi; spec at GET /openapi.json.
• GET /health route.
• Node server entry via @hono/node-server, plus worker.ts entry for the BullMQ worker.
• CI: build-api2 job in .github/workflows/web.yml.
• Flightcontrol domains: api2.rulebase.co, eu.api2.rulebase.co.

Data layer:

• Drizzle ORM + api2 Postgres schema. Migrations well past the original dozen (drizzle/0000_* … drizzle/0228_* as of this update).
• Synced mirrors of Rails: users, organizations, integrations, connections, conversations, memberships — all carry rulebase_id text unique.
• Large native surface: workflows + versions + runs/node executions, chats/artifacts, coaching, dashboards, work items, knowledge base, memory, credits/billing, onboarding tasks/cases, SLA outcomes, and more. See schema.ts.

Agent / workflow primitive:

• workflows with instructions, tools jsonb, triggers jsonb, optional graph, executionMode, schedule indexes; agent_versions for history.
• BullMQ + Redis job runner with separate worker process (worker.ts). Bull-Board UI at /queues.
• Schedule trigger evaluation via Croner; event triggers via Rails → api2 webhook forward.
• Agent execution via Vercel AI SDK (ai, @ai-sdk/openai) — Mastra not adopted.
• Braintrust integration for LLM observability.
• Tool catalog expanded beyond the original rulebase | slack | email set (see agent-tool.ts).

Rails bridge + query path:

• lib/rails-client.ts with cookie-auth session resolution + bearer-auth internal endpoints.
• query_data validator/executor on api2 (see query_data migration).
• Conversation QA evaluation pieces still span Rails and api2.

Chat / Instruct / MCP / credits:

• Chats + chat-messages + artifacts for in-product chat.
• MCP server (routes/mcp.ts) on mcp.rulebase.co / eu.mcp.rulebase.co.
• Credit metering via credit_usage_events (+ org billing settings / membership spend caps).

Not yet shipped / still deferred (from the original plan)

• Full Sheets + TipTap Pages port as written in those sections. Pages product surface removed; sheets only partially on api2.
• Inngest self-host. Deferred until BullMQ stops being enough.
• Mastra integration. Re-evaluate when a use case clearly benefits.
• Generic credential-fetch endpoint (GET /internal/organizations/:id/integration_credentials/:type) — optional; not required for every TS-native client.
• Uniform TS-native tool execution for every vendor — direction holds; coverage is uneven.
• Trace propagation (X-Rulebase-Trace-Id middleware end-to-end).
• Yjs / Hocuspocus for collaborative Pages — moot while Pages are gone; not active work.
• JWT-based service-to-service auth between Rails and api2 — currently uses a shared bearer key (RAILS_INTERNAL_API_KEY) for Rails→api2-side calls, and cookie/JWT passthrough for UI→api2 session resolution.

Sync-tier learnings (2026-05-11)

A SyncConversationJob production incident validated and refined two parts of this plan. Capturing the specifics here so the future sync-port project has the constraints written down rather than rediscovered.

What happened

PR #6388 added per-organization-data-source rate limiting to several Rails sync/import jobs via GoodJob’s derived_concurrency_rule (label-based concurrency, introduced in good_job 4.16 via #1700 ). Under webhook-driven fanout on a hot ODS label (sync-conv-ods-35, ~50K unfinished jobs in queue), the rule’s per-key pg_advisory_xact_lock serialized throttle checks across hundreds of workers, and the label-scoped count query ran inside the lock.

Production numbers from the incident:

• 200 throttle-failed executions sampled: p50 = 13.7s, p95 = 14.8s, max = 15.2s — tight distribution, structural cost, not load noise.
• Last hour at peak: 80,658 total SyncConversationJob executions, 953 succeeded, 74,964 throttled (93%). Effective throughput 16/min vs the 300/min the throttle was configured to allow.
• ~1M worker-seconds per hour spent on failed throttle checks for this one label, starving every other job class sharing the worker pool.

Why this happens

The throttle implementation is documented in GoodJob’s own README as “optimistic — assumes collisions are atypical.” The maintainer’s official guidance for high-collision workloads is to use thread/process concurrency instead. Issue #838  captures the same failure mode (high CPU when worker count > perform_limit at high queue depth) and has been open since 2023. It is not a bug; it is a documented scale limit of the underlying primitive.

Current Rails-side fix (PRs #6435, #6437)

1. Remove derived_concurrency_rule from hot-label sync jobs.
2. Re-add precise rate limiting via lib/rate_limiter.rb — a Redis fixed-window token bucket (atomic Lua, sub-millisecond, no DB locks). This is the same primitive BullMQ uses for its groupKey rate limiter.
3. Move SyncConversationJob to a dedicated GoodJob queue (sync_conversation:4) so one hot ODS can’t starve other job classes on the worker pool.

The two layers compose: Redis bucket is the precise per-ODS rate gate; queue pool is the blast-radius backstop.

What this means for the sync-port project

The “Non-goals” section above defers sync-port (Zendesk/Aircall/Salesforce/Intercom) to a later phase. When that phase happens:

• Use BullMQ’s native limiter: { max, duration, groupKey } for per-ODS API rate limiting. It’s the same shape as the Redis token bucket we ended up writing in Rails, but built in and integrated with the worker loop.
• Use per-queue concurrency for the blast-radius backstop. Same shape as the dedicated GoodJob queue pool.
• Do not attempt to port GoodJob’s good_job_concurrency_rule / derived_concurrency_rule abstraction. It is a known scale failure mode and the maintainer doesn’t intend to fix it.

What this does not mean

This is not by itself an argument for accelerating sync-port. The Rails-side fix is workable, and the strangler ordering (agent primitives first, sync second) remains right. The incident sharpens the design tenets for the eventual sync-port project; it does not change its phasing.

It also doesn’t argue against Postgres-as-queue in general. Other Postgres-backed queues (Que, Solid Queue) don’t have this specific failure mode — it’s a GoodJob feature-level choice, not an inherent Postgres limit. The BullMQ argument here is “right primitives for our scale,” not “Postgres-can’t-do-this.”