Qonto Evaluation Coverage - Week of 2026-06-22

Scope

• Customer: Qonto
• Organization ID: 33
• Week: 2026-06-22 through 2026-06-28 in Europe/Paris
• UTC query window: 2026-06-21 22:00:00 through 2026-06-28 22:00:00
• Cohort: active, human, evaluable Qonto employees with a team
• Evaluation target: 5 chat/email tickets per week per agent, 3 call tickets per week per agent
• Starting focus: 79 of 380 teamed/evaluable employees had zero active completed AI evaluations

This note is the focused investigation log for the week of 2026-06-22. The first section explains the 79 agents with zero evaluations. Later sections should cover the other quota-miss buckets, especially agents who had some evaluations but did not reach the chat/email or call targets.

An eligible candidate pair means one (employee, conversation) pair where:

• conversation_agents.employee_id = employee.id
• conversation_agents.eligible_for_qa_evaluation = true
• the conversation was resolved in the 2026-06-22 week
• the conversation had ticket_status = resolved

Recommendations and impact

The first list contains configuration choices for Qonto. The second list contains Rulebase-owned fixes.

Customer recommendations

1. Review the strict chat/email eligibility policy

For agents who missed the 5/week chat/email target, the largest exact blocker was Qonto’s custom AI/freeform eligibility policy. These are not Rulebase ingestion failures; they are policy outcomes from Qonto’s configured eligibility instructions.

Based on a production rerun on 2026-07-02 for the week of June 22:

2. Stop waiting for linked tickets to close

Today, some resolved tickets are not evaluated because another related ticket is still open. Turning this off would allow Rulebase to evaluate the resolved ticket without waiting for every linked ticket in the same customer issue to close.

Based on the week of June 22:

3. Extend the evaluation lookback window

Today, some tickets are skipped because the customer opened them more than 30 days before they were resolved. This can miss long-running tickets even when the agent resolved them during the target week.

Based on a production rerun on 2026-07-02 for the week of June 22:

Rulebase follow-ups

1. Fix call ticket channel detection before sampling

Today, some tickets clearly look like calls, but the channel value used by the coverage rules is blank. This is a Rulebase-side classification issue: those tickets can miss Qonto’s existing call sampling rule even though they are call conversations.

Based on the week of June 22:

2. Watch call coverage after the source-channel fix lands

Some call evaluations were manually marked inactive during cleanup, so the next week after rulebase-co/rulebase#7855  is merged and the call source-channel backfill is run should be treated as a monitoring week rather than a clean before/after read.

What to watch:

• Newly backfilled call tickets should enter Qonto’s existing call coverage rule instead of sitting outside sampling.
• Active completed call evaluations should rise with the fixed source-channel data.
• If sampling counters show agents at the call target but active completed call evaluations are still low, investigate inactive completed call rows and sampling-counter reconciliation before treating the remaining gap as low ticket volume.

3. Audit chat/email evaluations that completed but did not count

Some chat/email candidate pairs reached a completed evaluation state, but still did not count as active completed chat/email evaluations for the target agent in the effective week.

Based on a production rerun on 2026-07-02 for the week of June 22:

4. Review chat/email no_agents_sampled rows

Some requests passed earlier gates, but no agent was selected when the evaluation started. For agents who were still under the chat/email target, this should be checked against sampling counter state and agent partition selection.

Based on a production rerun on 2026-07-02 for the week of June 22:

Zero active completed AI evaluations

Zero-evaluation definition: no active, completed AI qa_agent_evaluations in the week.

Why the 79 zero-evaluation agents had no AI evaluations

The highest-impact finding is that 52 of the 79 zero-evaluation employees had no QA-eligible resolved candidate conversations at all:

• 44 had no resolved handled conversations in the week.
• 8 had resolved handled conversations, but every conversation-agent link was eligible_for_qa_evaluation = false.

For the remaining 27 employees, the largest downstream blocker was Qonto’s AI/freeform eligibility policy.

Samples below are representative rows, mostly sorted by candidate or handled volume. Conversation URLs are examples, not exhaustive.

No resolved handled conversations

These employees were active/evaluable and teamed, but had no resolved conversations attached through conversation_agents in the week. There are no example conversation URLs for this bucket because there were no resolved handled conversation links in the week.

AI/freeform ineligible

These employees had eligible resolved candidate conversations, but their candidate tickets were primarily rejected by Qonto’s AI/freeform eligibility instructions. The dominant themes seen in this bucket are Qonto-specific policy checks: multiple human agents on the same customer conversation, no meaningful two-way exchange, outbound-call exclusions, and authentication/identity flows where the underlying issue was not handled.

Resolved handled conversations, but none eligible for QA

These employees had resolved conversations attached through conversation_agents, but none of those conversation-agent links were QA eligible. The Kaba spot check showed mostly internal note parts rather than customer-facing chat/email/call parts, which explains why eligible_for_qa_evaluation stayed false.

Linked tickets still open

These employees had eligible resolved candidate conversations, but the primary reason no evaluation was produced was that related linked tickets were still open.

Note: Thouraya A. had a tie between AI/freeform ineligible and linked tickets still open, with two candidate pairs in each bucket. The existing 79-agent breakdown places this employee in linked tickets, giving the documented 12 AI/freeform employees and 6 linked-ticket employees.

No customer response

These employees had eligible resolved candidate conversations, but the requests were skipped because the conversation did not have a qualifying customer response.

Outside evaluation window

These employees had eligible resolved candidate conversations, but the requests were skipped as outside the evaluation window.

Qonto’s current evaluation_ticket_started_window_days is 30. The gate uses the ticket start time (started_at, falling back to external_created_at) relative to the time the QA cron writes the request/skip. For these eight skipped candidate pairs, the oldest ticket was Eya A.’s conversation_bwLn1Z8Rn28tE59MRQozA27e : it started on 2026-03-12 14:14:58 UTC and was skipped on 2026-06-24 12:03:48 UTC, 103.91 days later.

Minimum setting that would have let all eight pass when they were skipped last week: 104 days. If changing the setting after the fact and immediately replaying normal eligibility on 2026-07-02, the same oldest ticket is already 111.54 days old, so the immediate replay minimum is 112 days. A practical setting with some buffer is 120 days.

Not sampled

This employee had eligible candidate conversations, but the primary reason was sampling-rule gating rather than eligibility.

Call tickets missing source channel

Costas A.’s not-sampled tickets exposed a broader call-channel issue. The five affected tickets were call conversations, but their source channel was blank. Qonto’s call coverage rule looks for source channel call, so these tickets did not match the call rule even though they were calls.

This is a widespread issue for Qonto and it is still occurring. The weekly counts below use conversations with a call channel type; checking for at least one actual call part produced the same missing-source counts.

The affected value is blank, not the literal value unknown: the production check found zero source_channel = unknown rows in these weekly buckets.

Request completed, but not as an active completed eval for this agent

This employee had one eligible candidate where the evaluation request completed, but it did not result in an active completed AI agent evaluation for this specific subject employee in the report window.

Partial coverage: chat/email under target

After the 79 zero-evaluation agents, the next major coverage issue is partial coverage: agents who received some AI evaluations but still missed one or both weekly targets.

The original 2026-07-01 CSV showed 178 agents missing the chat/email target with a 647-evaluation gap. A direct production rerun on 2026-07-02 shows 177 agents and a 645-evaluation gap, so one agent appears to have gained two chat/email evaluations since the export.

For the week starting 2026-06-22, using the current production rerun:

The chat/email gap is smaller than the call gap, but still material. Of the 177 agents who missed the chat/email target:

Those 177 agents had 2,523 resolved QA-eligible chat/email candidate pairs across 125 agents. 52 of the 177 under-target agents had no resolved QA-eligible chat/email candidate pairs in the week. 97 had at least five such candidate pairs but still missed the 5/week chat/email target.

Exact blocker breakdown for the unresolved chat/email target gap:

Target-capped impact means the row is capped by each agent’s remaining chat/email target gap. For example, if an agent needed two more evaluations and had ten blocked tickets in the row, the impact is counted as two. These rows are not additive across recommendations because the same agent’s remaining gap can be filled by more than one blocker category.

AI/freeform policy themes within the 1,228 AI/freeform-blocked candidate pairs: