From 056b5eb89d41760a055662c2e893f9a6d446206b Mon Sep 17 00:00:00 2001 From: Vishnu Narayanan Date: Mon, 13 Jul 2026 18:15:25 +0530 Subject: [PATCH] fix: avoid full scan in IMAP email dedup on large inboxes (#14981) ## Description `Imap::BaseFetchEmailService#email_already_present?` used `find_by(source_id:)`, which inherits `Message`'s `default_scope { order(created_at: :asc) }`, adding an `ORDER BY created_at ASC LIMIT 1` to what is only a presence check. On inboxes with a large message history, that `ORDER BY` lets Postgres satisfy the sort by walking `index_messages_on_created_at` instead of the selective `index_messages_on_source_id`. For a not-yet-seen `source_id` (every new email) it can scan the whole table before returning, taking seconds per message. The dedup loop runs with no IMAP activity in between, so the idle socket is dropped by the mail server and the fetch job aborts with `closed stream`. The inbox then stops ingesting mail entirely, while smaller inboxes on the same server keep working. `exists?` issues `SELECT 1 ... LIMIT 1` with no `ORDER BY`, so the planner uses `index_messages_on_source_id` regardless of table size. No schema change is required. The fix lives in the shared base class, so it covers both the IMAP and Microsoft fetch paths. Fixes #14682 --- app/services/imap/base_fetch_email_service.rb | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/app/services/imap/base_fetch_email_service.rb b/app/services/imap/base_fetch_email_service.rb index e55355f3a..9c5b8e27b 100644 --- a/app/services/imap/base_fetch_email_service.rb +++ b/app/services/imap/base_fetch_email_service.rb @@ -38,7 +38,8 @@ class Imap::BaseFetchEmailService end def email_already_present?(channel, message_id) - channel.inbox.messages.find_by(source_id: message_id).present? || deleted_message_tracker.deleted?(message_id) + # exists? avoids Message's default_scope ORDER BY, which full-scans large inboxes + channel.inbox.messages.exists?(source_id: message_id) || deleted_message_tracker.deleted?(message_id) end def deleted_message_tracker