akadmin/chatwoot
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

fix: improve email inbox IMAP and SMTP compatibility (#14589)

Browse Source 
Fetch IMAP message content using `BODY.PEEK[]` instead of `RFC822` to
avoid provider-specific parser failures while preserving unread state.
This also applies the existing SMTP timeout configuration to custom SMTP
email-channel replies, so provider SMTP responses have enough time to
complete.

Fixes: https://github.com/chatwoot/chatwoot/issues/12762

## Why

Some IMAP providers can return responses for `FETCH RFC822` that Ruby
`net-imap` fails to parse with:

`Net::IMAP::ResponseParseError: unexpected RPAR (expected ATOM or NIL)`

We reproduced this with iCloud IMAP. Authentication, `INBOX` selection,
and header fetches worked, but fetching full message content with
`RFC822` failed before Chatwoot received a `Mail::Message`.

The same mailbox successfully returned full message content when fetched
with `BODY.PEEK[]`.

> During end-to-end iCloud validation, inbound fetch worked after the
IMAP change, but outbound replies through the custom SMTP settings could
still fail with a socket read timeout. The OAuth SMTP path already used
explicit SMTP timeout values; the custom SMTP path was relying on mailer
defaults instead.

## What this change does

- Replaces the full message fetch from `RFC822` to `BODY.PEEK[]`
- Reads the returned message content from `BODY[]`, which is how
`net-imap` exposes the response attribute
- Keeps the existing `BODY.PEEK[HEADER]` header-fetch behavior unchanged
- Applies `SMTP_OPEN_TIMEOUT` and `SMTP_READ_TIMEOUT` to custom SMTP
email-channel replies
- Defaults custom SMTP reply delivery to `open_timeout: 15` and
`read_timeout: 30`
- Updates IMAP service specs for standard and Microsoft IMAP fetch flows
- Updates mailer specs for custom SMTP timeout settings

`BODY.PEEK[]` is preferable here because it fetches the full message
content without marking messages as read.

## Validation

- Configured a local email inbox against iCloud IMAP and SMTP
- Confirmed `FETCH RFC822` reproduces `Net::IMAP::ResponseParseError:
unexpected RPAR (expected ATOM or NIL)`
- Confirmed `BODY[]` and `BODY.PEEK[]` fetch the same mailbox
successfully
- Confirmed Chatwoot imports iCloud messages after the IMAP change
- Sent two outbound replies from the Chatwoot UI through iCloud SMTP
after applying the timeout settings
- Confirmed both UI-created outbound messages were marked `sent`, had
iCloud SMTP `source_id` values, and had no `external_error`
- Ran `bundle exec rspec spec/services/imap/fetch_email_service_spec.rb
spec/services/imap/microsoft_fetch_email_service_spec.rb`
- Ran `bundle exec rspec spec/mailers/conversation_reply_mailer_spec.rb`
This commit is contained in:
Sojan Jose
2026-06-03 15:56:54 +05:30
committed by GitHub
parent 1beaa284c6
commit 8e42307bdc
5 changed files with 45 additions and 17 deletions
Download Patch File Download Diff File Expand all files Collapse all files
app/mailers/conversation_reply_mailer_helper.rb
+9 -2
View File
@@ -54,8 +54,7 @@ module ConversationReplyMailerHelper
tls: false,
enable_starttls_auto: true,
openssl_verify_mode: 'none',
open_timeout: 15,
read_timeout: 15,
**smtp_timeout_settings,
authentication: 'xoauth2'
}
end
@@ -72,6 +71,7 @@ module ConversationReplyMailerHelper
tls: @channel.smtp_enable_ssl_tls,
enable_starttls_auto: @channel.smtp_enable_starttls_auto,
openssl_verify_mode: @channel.smtp_openssl_verify_mode,
**smtp_timeout_settings,
authentication: @channel.smtp_authentication
}
@@ -79,6 +79,13 @@ module ConversationReplyMailerHelper
@options[:delivery_method_options] = smtp_settings
end
def smtp_timeout_settings
{
open_timeout: ENV['SMTP_OPEN_TIMEOUT'].presence || 15,
read_timeout: ENV['SMTP_READ_TIMEOUT'].presence || 30
}.transform_values(&:to_i)
end
def email_smtp_enabled?
@inbox.inbox_type == 'Email' && @channel.smtp_enabled
end
app/services/imap/base_fetch_email_service.rb
+3 -2
View File
@@ -58,8 +58,9 @@ class Imap::BaseFetchEmailService
return if email_already_present?(channel, message_id)
# Fetch the original mail content using the sequence no
mail_str = imap_client.fetch(seq_no, 'RFC822')[0].attr['RFC822']
# Fetch the original mail content using the sequence no.
# BODY.PEEK[] avoids RFC822 parser failures seen with some IMAP servers.
mail_str = imap_client.fetch(seq_no, 'BODY.PEEK[]')[0].attr['BODY[]']
if mail_str.blank?
Rails.logger.info "[IMAP::FETCH_EMAIL_SERVICE] Fetch failed for #{channel.email} with message-id <#{message_id}>."
spec/mailers/conversation_reply_mailer_spec.rb
+20
View File
@@ -462,6 +462,26 @@ RSpec.describe ConversationReplyMailer do
expect(mail.delivery_method.settings.empty?).to be false
expect(mail.delivery_method.settings[:address]).to eq 'smtp.gmail.com'
expect(mail.delivery_method.settings[:port]).to eq 587
expect(mail.delivery_method.settings[:open_timeout]).to eq 15
expect(mail.delivery_method.settings[:read_timeout]).to eq 30
end
it 'uses configured smtp timeout values' do
with_modified_env SMTP_OPEN_TIMEOUT: '10', SMTP_READ_TIMEOUT: '30' do
mail = described_class.email_reply(message)
expect(mail.delivery_method.settings[:open_timeout]).to eq 10
expect(mail.delivery_method.settings[:read_timeout]).to eq 30
end
end
it 'uses default smtp timeout values when env values are blank' do
with_modified_env SMTP_OPEN_TIMEOUT: '', SMTP_READ_TIMEOUT: '' do
mail = described_class.email_reply(message)
expect(mail.delivery_method.settings[:open_timeout]).to eq 15
expect(mail.delivery_method.settings[:read_timeout]).to eq 30
end
end
it 'renders sender name in the from address' do
spec/services/imap/fetch_email_service_spec.rb
+7 -7
View File
@@ -80,11 +80,11 @@ RSpec.describe Imap::FetchEmailService do
travel_to '26.10.2020 10:00'.to_datetime do
email_object = create_inbound_email_from_fixture('only_text.eml')
email_header = Net::IMAP::FetchData.new(1, 'BODY[HEADER]' => eml_content_with_message_id)
imap_fetch_mail = Net::IMAP::FetchData.new(1, 'RFC822' => eml_content_with_message_id)
imap_fetch_mail = Net::IMAP::FetchData.new(1, 'BODY[]' => eml_content_with_message_id)
allow(imap).to receive(:search).with(%w[SINCE 25-Oct-2020]).and_return([1])
allow(imap).to receive(:fetch).with([1], 'BODY.PEEK[HEADER]').and_return([email_header])
allow(imap).to receive(:fetch).with(1, 'RFC822').and_return([imap_fetch_mail])
allow(imap).to receive(:fetch).with(1, 'BODY.PEEK[]').and_return([imap_fetch_mail])
allow(imap).to receive(:logout)
result = described_class.new(channel: imap_email_channel).perform
@@ -93,7 +93,7 @@ RSpec.describe Imap::FetchEmailService do
expect(result[0].message_id).to eq email_object.message_id
expect(imap).to have_received(:search).with(%w[SINCE 25-Oct-2020])
expect(imap).to have_received(:fetch).with([1], 'BODY.PEEK[HEADER]')
expect(imap).to have_received(:fetch).with(1, 'RFC822')
expect(imap).to have_received(:fetch).with(1, 'BODY.PEEK[]')
expect(logger).to have_received(:info).with("[IMAP::FETCH_EMAIL_SERVICE] Fetching mails from #{imap_email_channel.email}, found 1.")
expect(imap).to have_received(:logout)
end
@@ -115,7 +115,7 @@ RSpec.describe Imap::FetchEmailService do
expect(result.length).to eq 0
expect(imap).to have_received(:search).with(%w[SINCE 25-Oct-2020])
expect(imap).to have_received(:fetch).with([1], 'BODY.PEEK[HEADER]')
expect(imap).not_to have_received(:fetch).with(1, 'RFC822')
expect(imap).not_to have_received(:fetch).with(1, 'BODY.PEEK[]')
end
end
@@ -129,12 +129,12 @@ RSpec.describe Imap::FetchEmailService do
Net::IMAP::FetchData.new(seq_num, 'BODY[HEADER]' => eml_content_without_message_id)
end
valid_email_header = Net::IMAP::FetchData.new(valid_message_seq_num, 'BODY[HEADER]' => eml_content_with_message_id)
imap_fetch_mail = Net::IMAP::FetchData.new(valid_message_seq_num, 'RFC822' => eml_content_with_message_id)
imap_fetch_mail = Net::IMAP::FetchData.new(valid_message_seq_num, 'BODY[]' => eml_content_with_message_id)
allow(imap).to receive(:search).with(%w[SINCE 25-Oct-2020]).and_return(empty_message_id_seq_nums + [valid_message_seq_num])
allow(imap).to receive(:fetch).with(empty_message_id_seq_nums, 'BODY.PEEK[HEADER]').and_return(empty_message_id_headers)
allow(imap).to receive(:fetch).with([valid_message_seq_num], 'BODY.PEEK[HEADER]').and_return([valid_email_header])
allow(imap).to receive(:fetch).with(valid_message_seq_num, 'RFC822').and_return([imap_fetch_mail])
allow(imap).to receive(:fetch).with(valid_message_seq_num, 'BODY.PEEK[]').and_return([imap_fetch_mail])
allow(imap).to receive(:logout)
result = described_class.new(channel: imap_email_channel).perform
@@ -143,7 +143,7 @@ RSpec.describe Imap::FetchEmailService do
expect(result[0].message_id).to eq email_object.message_id
expect(imap).to have_received(:fetch).with(empty_message_id_seq_nums, 'BODY.PEEK[HEADER]')
expect(imap).to have_received(:fetch).with([valid_message_seq_num], 'BODY.PEEK[HEADER]')
expect(imap).to have_received(:fetch).with(valid_message_seq_num, 'RFC822')
expect(imap).to have_received(:fetch).with(valid_message_seq_num, 'BODY.PEEK[]')
end
end
end
spec/services/imap/microsoft_fetch_email_service_spec.rb
+6 -6
View File
@@ -30,11 +30,11 @@ RSpec.describe Imap::MicrosoftFetchEmailService do
travel_to '26.10.2020 10:00'.to_datetime do
email_object = create_inbound_email_from_fixture('only_text.eml')
email_header = Net::IMAP::FetchData.new(1, 'BODY[HEADER]' => eml_content_with_message_id)
imap_fetch_mail = Net::IMAP::FetchData.new(1, 'RFC822' => eml_content_with_message_id)
imap_fetch_mail = Net::IMAP::FetchData.new(1, 'BODY[]' => eml_content_with_message_id)
allow(imap).to receive(:search).with(%w[SINCE 25-Oct-2020]).and_return([1])
allow(imap).to receive(:fetch).with([1], 'BODY.PEEK[HEADER]').and_return([email_header])
allow(imap).to receive(:fetch).with(1, 'RFC822').and_return([imap_fetch_mail])
allow(imap).to receive(:fetch).with(1, 'BODY.PEEK[]').and_return([imap_fetch_mail])
allow(imap).to receive(:logout)
result = described_class.new(channel: microsoft_channel).perform
@@ -45,7 +45,7 @@ RSpec.describe Imap::MicrosoftFetchEmailService do
expect(result[0].message_id).to eq email_object.message_id
expect(imap).to have_received(:search).with(%w[SINCE 25-Oct-2020])
expect(imap).to have_received(:fetch).with([1], 'BODY.PEEK[HEADER]')
expect(imap).to have_received(:fetch).with(1, 'RFC822')
expect(imap).to have_received(:fetch).with(1, 'BODY.PEEK[]')
expect(logger).to have_received(:info).with("[IMAP::FETCH_EMAIL_SERVICE] Fetching mails from #{microsoft_channel.email}, found 1.")
end
end
@@ -56,11 +56,11 @@ RSpec.describe Imap::MicrosoftFetchEmailService do
travel_to '26.10.2020 10:00'.to_datetime do
email_object = create_inbound_email_from_fixture('only_text.eml')
email_header = Net::IMAP::FetchData.new(1, 'BODY[HEADER]' => eml_content_with_message_id)
imap_fetch_mail = Net::IMAP::FetchData.new(1, 'RFC822' => eml_content_with_message_id)
imap_fetch_mail = Net::IMAP::FetchData.new(1, 'BODY[]' => eml_content_with_message_id)
allow(imap).to receive(:search).with(%w[SINCE 18-Oct-2020]).and_return([1])
allow(imap).to receive(:fetch).with([1], 'BODY.PEEK[HEADER]').and_return([email_header])
allow(imap).to receive(:fetch).with(1, 'RFC822').and_return([imap_fetch_mail])
allow(imap).to receive(:fetch).with(1, 'BODY.PEEK[]').and_return([imap_fetch_mail])
allow(imap).to receive(:logout)
result = described_class.new(channel: microsoft_channel, interval: 8).perform
@@ -71,7 +71,7 @@ RSpec.describe Imap::MicrosoftFetchEmailService do
expect(result[0].message_id).to eq email_object.message_id
expect(imap).to have_received(:search).with(%w[SINCE 18-Oct-2020])
expect(imap).to have_received(:fetch).with([1], 'BODY.PEEK[HEADER]')
expect(imap).to have_received(:fetch).with(1, 'RFC822')
expect(imap).to have_received(:fetch).with(1, 'BODY.PEEK[]')
expect(logger).to have_received(:info).with("[IMAP::FETCH_EMAIL_SERVICE] Fetching mails from #{microsoft_channel.email}, found 1.")
end
end