Zendesk 429 storm — Qonto (May 2026)

Summary

Qonto reported that we were hitting their Zendesk API limits. The Good Job dashboard showed a wave of ZendeskAPI::Error::NetworkError: status 429 discards across four job classes, and Sentry traces over the last 7 days revealed ~424K 429s out of ~4.2M Zendesk calls (~10% overall) for Qonto alone, dominated by SyncConversationJob (349K of those 429s, an 18.4% per-call failure rate).

Counter-intuitively, Qonto’s account-wide Zendesk budget was not exhausted. They are on Suite Enterprise with a raised account limit of 2,500 rpm (likely the High Volume API add-on), and we were measuring 17–33% headroom in the global bucket while the 429s were happening. The two endpoints we were 429-ing on do not have documented per-endpoint sub-limits.

Trace-level inspection revealed two distinct failure modes that co-exist and reinforce each other:

1. Steady-state load. A successful SyncConversationJob makes 5–10 Zendesk calls, of which 3–5 are individual /ticket_fields/{id} lookups. This is the baseline pressure that pushes us toward Zendesk’s burst threshold during peak hours. ~1.55M of the ~4.2M weekly calls are this kind of healthy-but-wasteful traffic.
2. Retry storm. When a sync 429s, our retry uses :polynomially_longer (the retry-after header is logged and discarded), so the same ticket re-fires too soon and 429s again, up to 5–10 attempts. A single failing ticket can produce hundreds-to-thousands of 429s in a few minutes. ~349K of the 424K 429s come from this pattern, concentrated into bursts that the customer actually feels.

The two compound: steady-state load creates the conditions where Zendesk’s spike-detection is one-bad-burst away from triggering; the retry storm provides that bad burst. Fixing only one leaves the other intact.

We also have no unified per-tenant concurrency cap across the four Zendesk-touching jobs (SyncConversationJob, ProcessZendeskWebhookEventJob, ImportZendeskConversationJob, SyncAgentIntegrationProfileJob), and our throttles are minute-shaped rather than second-shaped — both contribute to bursty fan-out.

Fix is sequenced into four PRs (plus two optional follow-ups). Retry-After ships first because it directly attacks the retry-storm burst pattern that customers actually feel; the ticket_fields cache ships second because it cuts the baseline that makes those bursts inevitable.

Telemetry (Sentry, 7 days, qonto9015 only)

Per-job

SyncConversationJob produces ~349K of the ~424K total 429s (~82%). Fixing this one job buys the lion’s share of relief.

~4.2M calls / 7 days ≈ ~420 calls/min on average — well under the 2,500 rpm budget. So the storm is shape, not volume.

Per-endpoint patterns

URLs aren’t normalised in Sentry, so each {id} is its own row. Aggregating by pattern:

• GET /api/v2/ticket_fields/{id} — by far the heaviest call volume. Top rows are 113K, 85K, 75K calls each on individual field IDs over 7 days, across ~30+ fields. Every ticket sync re-fetches every custom field individually.
• GET /api/v2/users/{id} — top two rows are individual user IDs hit 42K and 13K times in 7 days at 7%–15% 429 rate. Same agents being fetched over and over across ticket syncs. ~5K+ 429s on just two user IDs.
• GET /api/v2/tickets/{id} — long tail of specific tickets stuck in retry loops with ~100% 429 rate (e.g. 1K calls = 1K 429s per ticket, dozens of tickets like this). Perpetually-failing syncs that never recover before being discarded.

Per-trace breakdown — two co-existing failure modes

Drilling into individual SyncConversationJob traces (grouped by trace ID) shows two distinct shapes:

Successful trace b779b160 (52 minutes ago): 8 calls to complete one sync.

Half the calls are /ticket_fields/{id}. A second sampled successful trace (6fae8d19) showed 5 calls — same shape, smaller ticket.

Failed traces (5399469, 5224470, etc.): 1 call → 429 → discard. Then the retry path re-enters and hits the same wall.

The dichotomy:

These aren’t competing — they’re complementary. The retry storm is what causes acute 429 spikes for the customer; the steady-state load sets the baseline that makes those spikes inevitable during peak hours. Both fixes are needed; neither is sufficient on its own.

Root cause

The 429s are not a single bug, they’re the cumulative effect of six amplifiers. Each is grouped below by which failure mode it primarily drives — all six are present, but the ones tagged “retry storm” are what the customer felt acutely, and the ones tagged “steady state” are what set the conditions for the storm to ignite.

1. /ticket_fields/{id} is fetched per-field per-sync — steady state (highest-volume cause)

Conversation::Sync::Zendesk#zendesk_ticket_field caches in @ticket_field_cache, but the cache is per-instance — initialized fresh in Conversation::Sync::Zendesk#initialize for every sync:

def initialize(conversation, organization_data_source)
  super
  @user_cache = {}
  @organization_cache = {}
  @ticket_field_cache = {}     # <- per-instance, dies with the sync
  @user_search_cache = {}
end
 
def zendesk_ticket_field(field_id)
  @ticket_field_cache[field_id] ||= zendesk_client.ticket_fields.find!(id: field_id)
end

Conversation::Sync::Zendesk#sync_ticket_custom_fields walks every ticket.custom_fields entry and calls human_readable_custom_field_value, which always calls zendesk_ticket_field(field_id) to fetch the field’s custom_field_options for human-readable rendering — even though find_or_create_conversation_field would have already DB-cached the existence check just above:

def human_readable_custom_field_value(field_id, raw_value)
  return 'None' if raw_value.blank?
  field_data = zendesk_ticket_field(field_id)   # <- bypasses our DB cache, hits Zendesk
  ...
end

For a tenant like Qonto with ~30 custom fields × millions of syncs, this single path generates the bulk of the /ticket_fields/{id} traffic Sentry sees. Ticket field definitions change rarely (days/weeks), so this is wholly avoidable with a per-ods process or Rails.cache TTL.

2. N+1 user lookups per sync, no sideloading or batching — steady state

Conversation::Sync::Zendesk fetches users one at a time:

def zendesk_user(user_id)
  @user_cache[user_id] ||= zendesk_client.users.find!(id: user_id)
end

It calls zendesk_user for the requester, the assignee, every audit-event author, and every chat-message actor. For a long ticket this is easily 10+ /users/{id} calls per sync. None of these are sideloaded on the initial tickets.find!, even though Zendesk supports ?include=users,groups,organizations,brands. There is also no users/show_many batching for the audit-author case. The cache, like @ticket_field_cache, is per-instance, so the same agents (e.g. Qonto’s most active L1 reps) are re-fetched on every ticket sync — explaining why the Sentry top-2 user IDs each see tens of thousands of repeat calls in a week.

3. Webhook handlers fetch the same ticket up to 3x — steady state

ZendeskWebhook::Payload::TicketCommentAdded#process walks this path:

1. Base#zendesk_ticket_from_payload → tickets.find!(id, include: :metric_sets) (call 1)
2. Calls Conversation::Import::Zendesk.import → tickets.find!(id, include: :metric_sets) (call 2)
3. Import calls conversation.sync_later → Conversation::Sync::Zendesk#load_ticket_for_sync → tickets.find!(id) (call 3)

Three round-trips for the same ticket, on every comment-added / status-changed webhook, with no caching or pass-through.

4. Retry-After is logged but ignored — retry storm (the core mechanism)

ApplicationJob detects 429s and raises ZendeskRateLimitedError, but:

• It logs the retry-after header and then discards it — the value is never plumbed through.
• The retry_on uses wait: :polynomially_longer, which is independent of what Zendesk actually told us to wait.
• Result: we either retry too soon (re-429) or wait far longer than needed.

This explains the long tail of /tickets/{id} rows in Sentry with ~100% 429 rate — those individual tickets enter a sync → 429 → retry too soon → 429 cycle that never breaks within the 5–10 attempt budget, then discards. Each failing ticket is its own little DoS contribution.

5. No per-tenant concurrency cap across Zendesk jobs — both modes

Existing throttles only cap rpm per job class, not per Zendesk tenant:

When Qonto’s webhooks burst (a busy support hour), all four jobs can fire concurrently and each contributes its own pile of in-flight requests. Nothing in the system says “no more than N concurrent Zendesk requests for this ods, period.”

6. Minute-shaped throttle = sub-second bursts — both modes

good_job_control_concurrency_with(perform_throttle: [1, 0.5.seconds]) is “1 job per 0.5s” — but in practice GoodJob releases jobs as soon as the throttle window ticks, which means the first dozen jobs in a minute can fire near-simultaneously across worker threads. That’s exactly the burst pattern Zendesk’s spike detection penalises.

The combined effect is that during Qonto’s peak hours, our jobs pile in faster than the spike-detection windows reset, the per-sync work amplifies what each job costs, and the retry behaviour spins failing tickets in tight loops.

Methodology

1. Inventory the failing jobs

The Good Job dashboard showed a discard cluster. Across ~25 sampled discards, every error was a 429 on either /api/v2/tickets/{id} or /api/v2/users/{id} for qonto9015.zendesk.com. The attempts column was 5 or 10, meaning these had cycled through the full retry budget — sustained pressure, not a transient burst.

2. Code-review the throttle/retry stack

Walked the four job classes plus ApplicationJob to map current rate-limiting behaviour. Surfaced the table in Root cause §5 plus the Retry-After gap in §4.

3. Confirm Qonto’s plan and global rpm budget (rails console)

ods = OrganizationDataSource.find_by_prefix_id!('connection_…')  # qonto9015 zendesk
 
ods.zendesk_client.connection.get('/api/v2/account/subscription.json').body['subscription']

Returned plan_name: "Enterprise", pricing_model_revision: 7, help_desk_size: "1000-4999". Per Zendesk’s rate limits doc , Enterprise alone is 700 rpm, but a single API call confirmed Qonto is at the High-Volume tier:

resp = ods.zendesk_client.connection.get('/api/v2/users/me.json')
resp.headers.slice(
  'x-rate-limit', 'ratelimit-limit',
  'x-rate-limit-remaining', 'ratelimit-remaining',
  'ratelimit-reset'
)
# => { "x-rate-limit" => "2500", "ratelimit-limit" => "2500",
#      "x-rate-limit-remaining" => "2033", "ratelimit-remaining" => "2033",
#      "ratelimit-reset" => "42" }

So 2,500 rpm budget, ~470 used in the current minute (about 19%). Far from the cap. Global rpm is not the constraint.

4. Look for per-endpoint sub-limits (rails console)

Repeated the header dump on the actual endpoints we were 429-ing on:

resp = ods.zendesk_client.connection.get("/api/v2/tickets/#{ticket_id}.json")
# => x-rate-limit: 2500, x-rate-limit-remaining: 2064, ratelimit-reset: 16
 
resp = ods.zendesk_client.connection.get("/api/v2/users/#{user_id}.json")
# => x-rate-limit: 2500, x-rate-limit-remaining: 1695, ratelimit-reset: 2

Neither response carried a Zendesk-RateLimit-Endpoint (or zendesk-ratelimit-...-show) header. Per the Zendesk docs, these per-endpoint headers are surfaced only for endpoints that actually have a sub-limit. GET /tickets/{id} and GET /users/{id} have no documented sub-limit; they fall under the global 2,500 rpm bucket.

5. Read the docs for the missing piece

The Zendesk Rate Limits page  has this clause under “Account limit”:

Notwithstanding the limits specified in this document, the system might still limit requests if it detects an unusual spike in requests from all sources for the account, including internal product requests.

That’s the only explanation consistent with the headers-only view — sustained rpm well under the cap, no per-endpoint sub-limit on the failing endpoints, but persistent 429s during peak hours when Qonto’s own agents are also hammering their account. Our job patterns (no concurrency cap, minute-shaped bursts, redundant fetches) make us the first to be throttled when the spike-detection window narrows.

6. Pull per-job, per-endpoint breakdown from Sentry

Aggregating Sentry’s traces for qonto9015 over 7 days surfaced the Telemetry tables above. Three things became immediately obvious that weren’t visible from headers or the Good Job dashboard:

• /ticket_fields/{id} is the dominant call volume by a wide margin — invisible in the Good Job dashboard because the discards we sampled all happened to be on /tickets/{id} and /users/{id}.
• A few specific user IDs and ticket IDs account for most of the repeat calls — pointing squarely at the per-instance cache lifetime as the design flaw.
• A long tail of /tickets/{id} rows have ~100% 429 rates — individual tickets that enter a doom-loop rather than transient failures.

This step shifted the leading priority from “concurrency cap first” to “fix ticket_fields caching first.”

7. Drill into individual successful and failed traces

The aggregate per-endpoint view was suggestive but not conclusive about why the same job class produces both a steady wave of small-cost successes and a cluster of high-rate failures. Looking at individual SyncConversationJob trace IDs in Sentry made it obvious:

• Successful traces carried 5–10 sequential Zendesk calls (the trace 1 example is typical), most of them /ticket_fields/{id} and /users/{id} with no batching.
• Failed traces carried a single call → 429 → discard. The repeat-call shape we see for individual tickets in the per-endpoint view is the retry path re-entering the same dead-end, not one job making 1K calls.

This is what produced the “two co-existing failure modes” framing in the Summary. It also corrected two earlier missteps:

• “ticket_fields cache first” was wrong on its own because cache improvements don’t help during the actual incident bursts (failed traces don’t even get past their first call to read a field).
• “Honor Retry-After is enough” was also wrong because it doesn’t touch the steady-state load that creates the burst conditions in the first place.

Both fixes are needed; the order is now driven by which pain customers feel first (acute bursts → Retry-After) and which buys the most lasting relief on volume (ticket_fields cache).

Proposed fix

Four core PRs plus two optional follow-ups, all separately revertable. The order targets the acute customer pain first (retry-storm bursts) and the chronic baseline pressure second (steady-state per-sync volume).

PR 1 — Honor Retry-After and cap 429 retries (stops the acute bursts)

Post-deploy note (May 7). The first cut of this PR (#6278 ) was silently dead code for ProcessZendeskWebhookEventJob, SyncConversationJob, and SyncConversationProviderStatusJob — three of the four jobs that handle the bulk of Qonto’s traffic. See Post-mortem: PR 1 was dead code for three of the four Zendesk-touching jobs for the root cause and the follow-up fix in #6286 .

The smallest diff and the highest customer-perceptible impact. Two changes:

In app/jobs/application_job.rb:

• Carry the retry-after value on ZendeskRateLimitedError (attr_reader :retry_after); plumb it through the rescue_from path.

• Change retry_on ZendeskRateLimitedError, wait: :polynomially_longer, attempts: 10 to:

  retry_on ZendeskRateLimitedError,
    wait: ->(error) { (error.retry_after || 60).to_i.seconds },
    attempts: 3

Both knobs matter:

• Honoring Retry-After breaks the “retry too soon → re-429” loop, which is the actual mechanism turning 1 failure into 1K calls.
• Capping attempts: 3 (down from 10) bounds the worst case for a ticket that keeps failing. The cron-driven incremental/tickets sweep will pick it up later when load is lower; we don’t need each webhook event to retry 10 times.

In OrganizationDataSource#build_zendesk_client, extend the existing insert_callback so 429 responses also log the response body (Zendesk sometimes hints at which limit was breached, and we currently throw that away).

This change directly attacks the retry-storm failure mode: the long tail of /tickets/{id} rows in Sentry with ~100% 429 rate (1K calls = 1K 429s) collapses to 3 calls per failing sync. ~349K of the 424K weekly 429s are this pattern, so the impact on the Sentry headline numbers is large.

PR 2 — Cache ticket_fields per ods (cuts the steady-state baseline)

In Conversation::Sync::Zendesk (and any other site that calls zendesk_client.ticket_fields.find!), replace the per-instance @ticket_field_cache with a per-ods cache backed by Rails.cache and a 1-hour TTL:

def zendesk_ticket_field(field_id)
  Rails.cache.fetch(
    ['zendesk_ticket_field', @organization_data_source.id, field_id],
    expires_in: 1.hour
  ) { zendesk_client.ticket_fields.find!(id: field_id) }
rescue ZendeskAPI::Error::RecordNotFound
  Rails.logger.warn("Custom field #{field_id} not found in Zendesk")
  nil
end

Optionally, prime the cache by listing all ticket fields once per ods (zendesk_client.ticket_fields.all) and caching the result; subsequent find!s become local lookups.

Field definitions change rarely (days/weeks), so a 1h TTL is generous. With Qonto’s ~30 fields and ~3–5 /ticket_fields/{id} calls per successful sync, this drops the steady-state per-sync cost to 2–5 calls and cuts total Zendesk volume by ~50% based on the Sentry breakdown — which is what removes the baseline pressure that pushes us into Zendesk’s spike threshold during peak hours.

PR 3 — Sideload users + global user cache

Two parts, both targeting the /users/{id} rows in Sentry:

1. Sideload on tickets.find!. Change tickets.find!(id, include: :metric_sets) to tickets.find!(id, include: 'metric_sets,users,groups,organizations,brands') in both Conversation::Import::Zendesk and Conversation::Sync::Zendesk#load_ticket_for_sync. Update Conversation::Sync::Zendesk’s @user_cache, @organization_cache, and @ticket_field_cache (post-PR-2) to seed from the sideloaded payload before falling back to per-id fetches.
2. Promote @user_cache to a per-ods Rails.cache entry with a short TTL (5–15 min). The same agents are touched across thousands of tickets in close succession; per-instance caching wastes the locality.
3. users/show_many for residual audit authors. Collect distinct audit.author_id and event.author_id values across ticket.audits, fetch in one users/show_many?ids=... call, prime the cache, then run the existing per-event loop.

Directly addresses the top-429 user IDs (42K and 13K calls/week becoming ~1 per cache-TTL window).

PR 4 — Per-tenant Zendesk concurrency cap (defense in depth)

Add a shared concurrency block to all four jobs:

good_job_control_concurrency_with(
  key: -> { "zendesk-#{zendesk_ods_id}" },
  perform_limit: 10  # tune
)

Where zendesk_ods_id resolves to the Zendesk organization data source id from each job’s arguments. The cap is per tenant, across all job classes, which is the missing piece today. Start at 10 in-flight requests per ods, dial up if we under-utilise.

While we’re here, also flip SyncConversationJob and SyncAgentIntegrationProfileJob from [1, 0.5.seconds] to [1, 1.second] so the throttle is genuinely per-second rather than minute-bursty.

After PRs 1–3, this is more about preventing future spikes (and protecting other tenants from a single noisy one) than fixing today’s storm.

PR 5 (optional) — Webhook ticket fetch dedupe

ZendeskWebhook::Payload::TicketCommentAdded#process walks Base#zendesk_ticket_from_payload → Conversation::Import::Zendesk.import → Conversation::Sync::Zendesk#load_ticket_for_sync, doing three tickets.find! calls for the same ticket. Pass the ticket object through, or have Conversation::Sync::Zendesk accept a pre-fetched ticket so load_ticket_for_sync skips its own fetch when called from the webhook path.

Nice cleanup but ~3x reduction on a smaller pool than ticket_fields or users. Ship if PRs 1–4 don’t fully close the gap.

PR 6 (optional) — Proactive backoff in the Zendesk client

In OrganizationDataSource#build_zendesk_client’s insert_callback, when x-rate-limit-remaining < 200 (or < 8% of x-rate-limit), sleep(ratelimit-reset). Cheap, safe, and respects the spike windows automatically.

What we deliberately won’t do

• Negotiate a higher Zendesk limit. Zendesk’s docs say increases above 2,500 rpm require prior written consent and an additional fee. We don’t need a higher cap; we’re nowhere near it. Asking would mask the real problem.
• Drop subscriptions just to reduce traffic. user.created, user.name_changed, etc. each cost a /users/{id} call, but the right fix is sideloading and batching, not dropping integration features.
• Disable Qonto’s sync. Customer’s actively using the product; this is a bug we own.
• Switch to bulk endpoints. incremental/tickets is already used for the sweep job; ad-hoc syncing per webhook is the right shape and just needs to be cheaper per call.

Verification

Re-run the same Sentry per-job and per-endpoint queries 24h after each PR deploys, and compare against the Telemetry baseline.

PR 1 — Honor Retry-After + cap retries

• Long-tail /tickets/{id} rows in Sentry with ~100% 429 rate disappear. Specific tickets that were stuck (e.g. ones with 1K calls = 1K 429s) now succeed within their 3-attempt budget — or, if they truly can’t, contribute at most 3 calls instead of 1K.
• Total 429 count for Qonto drops by the largest single increment of any PR (target: roughly 70%+ reduction, since the retry-storm pattern is ~349K of the 424K weekly 429s).
• Discarded SyncConversationJob / ImportZendeskConversationJob count drops materially (some discards remain because attempts: 3 is intentionally low; the cron-driven sweep is expected to re-pick them up).
• 429 log entries now include response bodies — useful for any future incident on a different tenant.

PR 2 — ticket_fields caching

• Total Zendesk call volume for Qonto drops by ~50% (Sentry baseline ≈ 4.2M/week → target around 2M/week), regardless of what 429 rate was after PR 1.
• /ticket_fields/{id} rows drop to roughly (distinct field count) × (cache misses per ods per hour) — dozens of calls per hour, not tens of thousands per field per week.
• Average calls per successful SyncConversationJob trace drops from 5–10 to 2–5.

PR 3 — Sideload + global user cache + users/show_many

• /users/{id} Sentry rows drop dramatically. Top user IDs that were hit 42K and 13K times/week should drop to single-digit calls per cache-TTL window.
• Average /users/{id} calls per SyncConversationJob (visible in trace breakdown) drops from ~3–5 to under 1.

PR 4 — Per-tenant concurrency cap

• During Qonto’s next business-hours peak (Paris time, weekday morning), GoodJob running count for Zendesk jobs stays at-or-below the chosen concurrency cap per ods.
• Residual 429 rate (post PRs 1–3) drops to near-zero.
• Job backlog (scheduled count for Zendesk jobs) does not grow unboundedly — if it does, the cap is too low and we dial up.
• No collateral impact on other tenants during a Qonto burst.

Aggregate target

After all four PRs:

• Total Qonto Zendesk call volume cut by ~70–80% (4.2M/week → under 1M/week).
• Overall 429 rate under 1% (baseline: ~10%).
• SyncConversationJob 429 rate under 1% (baseline: 18.4%).
• No ZendeskAPI::Error::NetworkError discards in any 24h window.

Customer-facing check

• Reach out to Qonto’s contact 48 hours after PRs 1 + 2 ship to confirm they’re no longer seeing rate-limit alerts on their side. PR 1 alone should be visible to them within the same day.

Status (May 7, 2026)

Shipped

Not yet started

• PR 3 — Sideload users + global user cache
• PR 4 — Per-tenant Zendesk concurrency cap
• PR 5 (optional) — Webhook ticket fetch dedupe
• PR 6 (optional) — Proactive backoff in the Zendesk client
• Follow-up — Audit retry_on StandardError catch-alls in Zendesk-adjacent jobs (SyncAgentIntegrationProfileJob, SyncCompanyIntegrationProfileJob, SyncClientIntegrationProfileJob, SyncCompanyJob, SyncClientJob); see post-mortem below for why this is split out

Post-mortem: PR 1 was dead code for three of the four Zendesk-touching jobs

After PR 1 deployed and the webhook pause was lifted on the morning of May 7, the GoodJob dashboard immediately showed ProcessZendeskWebhookEventJob 429-ing at 100% on the webhook drain — the exact pattern PR 1 was supposed to fix. SyncConversationJob showed zero calls in the same window, but only because it hadn’t run yet; same bug.

Root cause

ApplicationJob declares rescue_from ZendeskAPI::Error::NetworkError to convert 429s into ZendeskRateLimitedError so retry_on ZendeskRateLimitedError can honor Retry-After. Three child jobs already declared their own handlers for the same exception class:

• ProcessZendeskWebhookEventJob:32 — retry_on ZendeskAPI::Error::NetworkError, wait: 10.minutes, attempts: 5
• SyncConversationJob:34 — same line
• SyncConversationProviderStatusJob:23-24 — same line, plus an extra one for ZendeskAPI::Error::ClientError

ActiveSupport::Rescuable iterates rescue_handlers in reverse-registration order, so the children’s handlers ran first and short-circuited the parent’s chain. Every 429 in those jobs was being re-queued at a flat wait: 10.minutes, attempts: 5, ignoring Retry-After entirely. PR 1’s whole rescue→raise→retry chain was unreachable for the three jobs that actually handle the bulk of Qonto’s traffic.

ImportZendeskConversationJob was the only one of the four that worked as intended, because it never declared its own override — and that’s also the only job PR 1’s spec covered, which is why the regression slipped through.

Why this should have been caught

• The PR 1 spec only exercised the one job that didn’t have an override. False-confidence by spec selection.
• We had no structural assertion that “no ApplicationJob descendant shadows the Zendesk rescue chain”. Behavioural specs alone can’t catch a class of regression where the absence of a handler is what matters.
• Code review missed that the ApplicationJob change required cleanup at every child override site, because the parent change was framed as “add a handler” rather than “centralise handling — remove now-redundant child overrides”.

Fix (#6286 )

• One-line deletion of the shadowing handler in each of the three jobs (ProcessZendeskWebhookEventJob, SyncConversationJob, SyncConversationProviderStatusJob); each removed line is replaced with a comment explaining why you don’t add one back.
• Behavioural spec for each of the three jobs: simulate a 429 and assert ZendeskRateLimitedError is raised carrying the Retry-After value. These would have failed on main and are the coverage PR 1 was missing.
• Structural regression spec (spec/jobs/zendesk_error_handling_chain_spec.rb) that eager-loads all jobs and fails fast if any ApplicationJob descendant declares its own handler for ZendeskAPI::Error::NetworkError/::ClientError/::Error.

Adjacent gap — deliberately not in #6286

A handful of jobs (SyncAgentIntegrationProfileJob, SyncCompanyIntegrationProfileJob, SyncClientIntegrationProfileJob, SyncCompanyJob, SyncClientJob) declare retry_on StandardError, wait: 10.minutes as a generic backstop. StandardError is an ancestor of ZendeskAPI::Error::NetworkError, so that pattern shadows the parent chain in exactly the same way for any of those jobs that touch Zendesk — SyncAgentIntegrationProfileJob is the most obvious offender (named in the original investigation as one of the four Zendesk-touching jobs).

The structural regression spec deliberately does not flag this pattern. Some of those catch-alls intentionally re-raise after marking a record failed; deleting them blindly would mean transient DB / network errors stop being retried at all. Each one wants a per-job decision (replace with specific retry_ons, or restructure the catch-all to forward ZendeskAPI::Error::NetworkError to the parent chain). Tracked as a follow-up; ranked below the original PRs 3–4 because the worst-affected jobs (SyncConversationJob etc.) are now covered by #6286.

Lesson for next time

When adding a new rescue_from / retry_on to ApplicationJob, make the same PR audit (and remove or document) every existing child-class handler for the same exception class or any of its ancestors. The structural spec from #6286 now enforces this for the Zendesk chain; the same pattern should be reused for any future centralised job error handling.