Rails AI Request Inventory

Last audited: 2026-06-20

This inventory tracks AI requests still initiated from rulebase-api Rails. It includes direct OpenAI SDK calls, RubyLLM calls, OpenAI file/vector-store API calls, and Rails-owned calls into Rulebase/Modal ML services. It does not treat plain email template rendering as model inference.

How This Was Audited

Primary searches:

OPENAI_CLIENT, DIRECT_OPENAI_CLIENT, OpenAI::Client.new, OpenAI::Client.for_prompt
responses.create, responses.stream, chat.completions.create, audio.speech.create, embeddings.create
RubyLLM.chat, RubyLLM::Chat.new
files.create, files.delete, vector_stores.*
TRANSCRIBE_FN, TRANSLATE_FN, REDACT_FN, REDACT_MULTI_TURN_BATCH_FN, EMBED_FN
Rulebase::TopicModeling

Shared clients and wrappers:

rulebase-api/config/initializers/openai.rb defines DIRECT_OPENAI_CLIENT against https://api.openai.com/v1 and OPENAI_CLIENT against OpenaiConfig.new.uri_base.
rulebase-api/lib/open_ai/client.rb adds OpenAI::Client.for_prompt, a Helicone-wrapped client with prompt/environment headers.
rulebase-api/lib/open_ai/realtime_client.rb defines a Realtime WebSocket client, but no Rails call sites were found outside that wrapper.
rulebase-api/config/initializers/ruby_llm.rb points RubyLLM at the same OpenAI-compatible base URL.
rulebase-api/config/initializers/rulebase.rb defines Modal-backed TRANSCRIBE_FN, TRANSLATE_FN, REDACT_FN, REDACT_MULTI_TURN_BATCH_FN, and EMBED_FN.
rulebase-api/config/initializers/assembly_ai.rb defines ASSEMBLY_AI, but no Rails call sites were found for it.

Migration Priority

QA evaluation and feedback retrieval: highest volume, customer-facing, and already shaped like API2 agent/run-step work.
Rails agent loops: Sentinel, workflow agents, compliance risk-factor generation, tests/sheets, marketing review.
Translation/transcription/redaction/chunk embedding service calls: Rails still orchestrates them, even when model work is behind Modal services.
Knowledge-base/vector-store lifecycle and retrieval: large surface area, important dependency for QA and agents.
Reports, digests, simulations, and enrichment jobs: useful to migrate, but easier to batch behind stable API2 endpoints.
Deprecated/demo/legacy helpers: prefer pruning before migrating where possible.

1. QA Evaluation And Feedback

Area	Rails call sites	Request type	Notes
Conversation-level QA runner	`app/services/evaluations/conversation_qa_evaluation_runner.rb`	`OPENAI_CLIENT.responses.create` with tools and follow-up tool outputs	Main customer-facing QA evaluation loop. Uses org evaluation model, reasoning effort, flex service tier, metadata, and tool budget enforcement.
Scorecard criterion evaluation	`app/models/qa_agent_evaluation_scorecard_result.rb`	`OPENAI_CLIENT.responses.create` for initial/follow-up leaf and parent criterion evaluation	Handles KB tool loops and structured `AgentCriterionResult` / `AgentParentCriterionResult`.
Criterion-level human feedback expansion	`app/models/qa_agent_evaluation_scorecard_criterion_result.rb`, `app/models/evaluation_scorecard_criterion_result_comment.rb`	`OPENAI_CLIENT.responses.create`	Generates expanded comments from human corrections for future evaluation guidance.
Evaluation-level human feedback expansion	`app/models/qa_agent_evaluation.rb`	`OPENAI_CLIENT.responses.create`	Generates expanded feedback from corrected full QA evaluations.
Feedback vector store upload	`app/models/evaluation_scorecard_criterion_result_comment.rb`, `app/models/feedback_vector_store.rb`, `app/models/organization.rb`	`files.create`, `vector_stores.files.create/delete`, `vector_stores.create/delete`	Stores expanded feedback in OpenAI vector stores.
Feedback chunk embeddings	`app/models/evaluation_scorecard_criterion_result_comment/chunk_embedding_sync.rb`, `app/models/qa_agent_evaluation_scorecard_criterion_result/chunk_embedding_sync.rb`	`EMBED_FN.run`	Modal embedding service creates local `DocumentChunk` rows.
QA retrieval tools	`app/tool_calls/qa_evaluation/ask_feedback_store.rb`, `app/tool_calls/qa_evaluation/ask_knowledge_base.rb`, `app/tool_calls/qa_evaluation/ask_feedback_vector_store.rb`	`embeddings.create` or `KnowledgeBase.ask`	Mixed local vector search and OpenAI file-search vector-store retrieval.
Score option justification generation	`app/models/scorecard_criterion_score_option_justification_option.rb`	`RubyLLM.chat(...).ask`	Generates justification options from score options.

2. Translation, Transcription, Redaction, Embedding Services

Area	Rails call sites	Request type	Notes
Conversation translation	`app/models/conversation/translation_event.rb`, `app/jobs/translate_conversation_job.rb`	`TRANSLATE_FN.run`	Rails builds utterance/key-phrase payloads, calls Modal translation, writes `text_english` / `body_english`, and tracks credits.
Call transcription	`app/models/conversation_part_call_detail/transcription/service.rb`, `app/jobs/transcribe_conversation_part_call_detail_job.rb`, `app/jobs/transcribe_conversation_job.rb`, `app/jobs/process_qa_evaluation_request_job.rb`	`TRANSCRIBE_FN.run`	Provider currently passed as `elevenlabs`; Rails manages recordings, expected languages, transcript messages, credits, and downstream CX risk.
Single-text redaction	`app/models/conversation.rb`, `app/models/organization/redaction.rb`	`REDACT_FN.run`	Presidio-backed service call for PII redaction.
Multi-turn/batch redaction	`app/models/conversation/redaction_event.rb`	`REDACT_MULTI_TURN_BATCH_FN.run`	Handles call transcript messages plus other text parts in one redaction request.
Chunk embeddings	`app/models/knowledge_base_document/chunk_embedding_sync.rb`, `app/models/evaluation_scorecard_criterion_result_comment/chunk_embedding_sync.rb`, `app/models/qa_agent_evaluation_scorecard_criterion_result/chunk_embedding_sync.rb`	`EMBED_FN.run`	Produces chunk embeddings for local nearest-neighbor search.
Topic modeling	`app/models/organization.rb`, `app/models/complaint_report.rb`	`Rulebase::TopicModeling.analyze`	Clusters unassigned product/service issues and complaints.

3. Knowledge Base And OpenAI Vector Stores

Area	Rails call sites	Request type	Notes
Vector-store creation/deletion	`app/models/organization.rb`, `app/models/knowledge_base.rb`, `app/models/framework.rb`, maintenance tasks	`vector_stores.create/delete`	Creates KB, feedback, framework, and compliance-policy stores.
KB query	`app/models/knowledge_base.rb`, `app/controllers/knowledge_base_responses_controller.rb`, `app/models/framework.rb`, `app/models/organization.rb`, `app/models/compliance_profile.rb`, `app/models/workflow_node/run/ask_knowledge_base.rb`, `app/tool_calls/compliance_agent/search_framework_documents.rb`, `app/tool_calls/marketing_review/search_framework_documents.rb`, `app/models/sentinel_conversation_message/tools/search_policies.rb`	`responses.create` / `responses.stream` with `file_search`	Central wrapper used by QA, compliance agents, Sentinel tools, workflow KB tools, and direct KB responses.
KB document upload	`app/models/knowledge_base_document.rb`, `app/models/procedure.rb`, `app/models/framework_document.rb`	`files.create`, `vector_stores.files.create/delete`, `files.delete`	Syncs KB/procedure/framework content to OpenAI vector stores.
Compliance policy upload	`app/models/compliance_policy.rb`, `app/jobs/delete_compliance_policy_from_openai_job.rb`	`files.create/delete`, `vector_stores.files.create/delete`	Uploads policy files, links them to compliance policy vector store, and extracts titles.
KB crawler extraction	`app/models/knowledge_base_crawl_task.rb`, `app/models/framework_document/scraper.rb`	`OpenAI::Client.for_prompt(...).responses.create`	Extracts article content from raw HTML and decides crawl links.
Vector-store maintenance	`app/tasks/maintenance/rebuild_knowledge_base_vector_store_task.rb`, `backfill_knowledge_base_document_*` tasks	`vector_stores.create`, `vector_stores.files.update`	Rebuilds/stamps vector store metadata.

4. Compliance, Risk, Complaints, Disputes

Area	Rails call sites	Request type	Notes
CX risk snapshots	`app/models/conversation_cx_risk_snapshot.rb`, `app/models/conversation.rb`	`OpenAI::Client.for_prompt(...).responses.create`, `embeddings.create`	Conversation risk summary/classification and embeddings.
Complaint detection/follow-up	`app/models/evaluation_check/checks/complaint.rb`	`responses.create`	Checks whether a conversation contains a complaint and whether it follows up an existing complaint.
Customer issue detection/assignment	`app/models/evaluation_check/checks/customer_issue.rb`, `app/models/customer_issue.rb`, `app/models/conversation.rb`	`responses.create`, `embeddings.create`	Generates issue summaries, embeds descriptions, labels clusters, and assigns conversations to issue categories.
Complaint/customer issue themes	`app/models/complaint_theme_snapshot.rb`, `app/models/customer_issue_theme_snapshot.rb`, report generators	`responses.create`	Labels clusters and writes theme snapshot analysis.
Compliance rules and criteria	`app/models/compliance_rule_document.rb`, `app/models/compliance_criterion_group.rb`, `app/models/compliance_review.rb`, `app/models/compliance_review/upload_file/plain_text.rb`	`responses.create`	Checks rules, reviews criterion groups, extracts text from uploaded plain text, summarizes compliance reviews.
Compliance agent	`app/models/compliance_agent.rb`, `app/tool_calls/compliance_agent/query_program_documents.rb`	`responses.create`, `responses.stream`, `files.create/delete`	Generates program setup, risk factors, test question polarity, uploads generation files, and queries program documents.
Compliance policies	`app/models/compliance_policy.rb`	`responses.create`, `files/vector-store APIs`	Extracts policy titles and stores policy documents.
Disputes	`app/models/dispute.rb`	`OpenAI::Client.for_prompt(...).responses.create`	Checks dispute acknowledgement in transcript.
Scoring guides	`app/models/scoring_guide.rb`	`responses.create`, `files.create/delete`	Rates conversations against scoring guides, optionally with uploaded content files.
Customer sentiment	`app/models/conversation_customer_sentiment_summary.rb`	`OPENAI_CLIENT.responses.create`	Calculates sentiment score and summary for notifications.
QA risk summaries	`app/models/organization_risk_level_configuration.rb`	`responses.create`	Summarizes QA risk from score/results/transcript.
Knowledge gap reports	`app/models/agent_knowledge_gap_report.rb`, `app/models/workflow_node/run/agent/tools/knowledge_base_gaps.rb`	`responses.create`	Generates weekly knowledge gaps from evaluation evidence.

5. Rails Agents, Workflows, Sheets, Tests, Marketing Review

Area	Rails call sites	Request type	Notes
Sentinel conversations	`app/models/sentinel_conversation.rb`, `app/models/sentinel_conversation_message.rb`, `app/jobs/upload_sentinel_conversation_message_attachment_to_openai_job.rb`	`conversations.create`, `responses.create/stream`, `files.create`	Direct streaming agent loop with tools and OpenAI conversation state.
Sentinel helpers	`app/models/sentinel_report.rb`, `app/models/sentinel_conversation_message/functions/check_customer_request.rb`, `preview_upload.rb`, `tools/search_policies.rb`	`chat.completions.create`, `KnowledgeBase.ask`	Report generation, upload parsing, request checking, and policy search.
Workflow execution	`app/models/workflow_run.rb`, `app/models/workflow_node/run/agent.rb`	`RubyLLM::Chat.new`, `RubyLLM.chat`, `responses.create`	Old and new workflow agent runtimes still run from Rails.
Jira workflow nodes	`app/models/workflow_node/run/create_jira_issue_comment.rb`, `create_jira_issue_from_ticket.rb`	`OPENAI_CLIENT.responses.create`	Generates Jira comments and issue payloads from ticket context.
Workflow email ingestion	`app/mailboxes/workflow_mailbox.rb`	`OPENAI_CLIENT.responses.create`	Generates synthetic content for workflow-trigger emails.
Marketing review	`app/models/marketing_review.rb`	`files.create/delete`, `responses.create` with tools	File/image review agent loop.
Test report generation	`app/models/test_report.rb`	`DIRECT_OPENAI_CLIENT.responses.stream`	Streams report generation with tools.
Test row and sheet fill	`app/models/test_row.rb`, `app/models/sheet.rb`	`responses.create` with tools	Fills test/sheet columns with LLM tool loops.
Image analysis	`lib/image_analysis/vision_analyzer.rb`, `app/models/conversation_part.rb`	`responses.create` with image inputs	Generic workflow vision analysis and image caption generation.
Page AI write	`app/models/page.rb`	`OPENAI_CLIENT.responses.create`	Generates page text edits.

6. Simulations And Voice/Call Demos

Area	Rails call sites	Request type	Notes
Simulation runtime	`app/models/simulation.rb`	`conversations.create`, `responses.create`, `responses.stream`, `embeddings.create`, `files.create`	Large direct surface: persona/scenario/setup generation, simulated conversation, agent feedback, report evaluation, scorecard criterion evaluation, KB embeddings, matched ticket planning/distillation, document scenario handling.
Simulation setup run	`app/models/simulation_generate_setup_run.rb`	`OPENAI_CLIENT.responses.create`	Generates combined setup payload with structured schema.
Call scenario personas	`app/models/call_scenario_persona.rb`	`audio.speech.create`, `chat.completions.create`	Generates sample introduction text and OpenAI TTS audio.
Demo conversation generator	`lib/demo_conversation_generator.rb`	`DIRECT_OPENAI_CLIENT.chat.completions.create`	Generates demo conversations.

7. Reports And Digests

Area	Rails call sites	Request type	Notes
Report generator base	`app/models/report/generators/base.rb`	`OPENAI_CLIENT.responses.create`	Shared helper for report generators.
Report generators	`agent_performance.rb`, `compliance.rb`, `customer_insights.rb`, `complaint_theme_snapshot.rb`, `customer_issue_theme_snapshot.rb`, `weekend_recap.rb`	`responses.create`	Compliance/customer insights generators are marked deprecated in favor of `DigestEmail`, but still contain direct calls.
Agent performance reports	`app/models/agent_performance_report.rb`	`OPENAI_CLIENT.responses.create`	Generates report summary, trends, and coaching content.
Digest agent runner	`app/models/digest_email/agent_runner.rb`	`RubyLLM.chat(...).with_tools(...).ask`	Customer insights digest agent with tool calls to create/read digest data.
Digest supervisor review	`app/models/digest_email/supervisor_review.rb`	`OpenAI::Client.for_prompt(...).responses.create`	Reviews generated digest content and returns JSON verdict.
Digest grouping and quotes	`app/models/digest_email/issue_category_grouper.rb`, `app/models/digest_email/tools/get_conversation_details.rb`	`RubyLLM.chat(...).ask`	Generates category labels and relevant customer quotes.

8. Conversation Enrichment, Filters, And Legacy Helpers

Area	Rails call sites	Request type	Notes
Language detection	`app/models/conversation.rb`	`OpenAI::Client.new.responses.create`	Detects language from conversation text. Uses a direct `OpenAI::Client.new`, not the shared configured constants.
Call script adherence and agent summary	`app/models/conversation.rb`	`OPENAI_CLIENT.responses.create`	Reviews call-script adherence and generates agent-facing summary.
Conversation context and live summary	`app/models/conversation.rb`	`chat.completions.create`	Previous-conversation context and live summary/action suggestions.
Related conversation and issue assignment	`app/models/conversation.rb`	`OpenAI::Client.new.responses.create`	Identifies follow-up conversations and selects best customer issue.
Jira issue draft suggestions	`app/models/conversation.rb`	`OpenAI::Client.new.responses.create`	Suggests Jira workflow actions from ticket transcript.
Conversation filters	`app/models/conversation_filter.rb`	`OPENAI_CLIENT.responses.create`	Generates filter names and checks whether conversations match filters.
Call and call script legacy models	`app/models/call.rb`, `app/models/call_script.rb`, `app/models/call_script_guideline.rb`	`chat.completions.create`, `embeddings.create`	Risk summaries, script previews, and guideline embeddings. `CallScript` is deprecated.
Risk/social summaries	`app/models/customer_account.rb`, `app/models/location.rb`, `app/models/social_media_post.rb`	`chat.completions.create`	Legacy risk/social monitoring summaries.
Braintrust smoke test	`lib/tasks/test_braintrust.rake`	`chat.completions.create`	Rake task only.

Cleanup Notes

Several paths still use chat.completions.create; new migrations should probably land on API2 Responses/AI SDK patterns instead of preserving Chat Completions.
Conversation, ConversationPart, Sheet, and TestRow instantiate direct clients (OpenAI::Client.new or local constants) instead of consistently using Rails’ configured OPENAI_CLIENT/DIRECT_OPENAI_CLIENT.
ASSEMBLY_AI is configured but appears unused by Rails.
OpenAI::RealtimeClient is defined but appears unused by Rails.
Deprecated surfaces worth pruning before migration: CallScript, legacy Report::Generators::Compliance, and legacy Report::Generators::CustomerInsights.
OpenAI vector-store/file lifecycle is deeply coupled to Rails models. Moving generation to API2 without deciding vector-store ownership will leave Rails with many non-generation OpenAI API calls.