Files
chatwoot/app/mailboxes/mailbox_helper.rb
T
4371793741 fix: captain-v2 cannot see image attachments shared via email (#14449)
## Description

Inbound email attachments are stored with `file_type: 'file'` regardless
of their actual MIME type. As a result, image screenshots shared by
customers via email are not exposed to Captain V2's multimodal pipeline
— `Captain::OpenAiMessageBuilderService#attachment_parts` selects images
via `attachments.where(file_type: :image)` and emits a placeholder
`"User has shared an attachment"` text part instead of an `image_url`
part. The model never gets the image, so Captain keeps asking the
customer to retype information that is already visible in the
screenshot.

This PR makes the email mailbox derive `file_type` from the blob's
`content_type` using the existing shared `FileTypeHelper`, matching how
every other inbound channel (`twilio`, `sms`, `telegram`, `line`,
`tiktok`, `twitter`, `messenger`) and `MessageBuilder` already classify
attachments.

Fixes #14448

## Type of change

- [x] Bug fix (non-breaking change which fixes an issue)

## How Has This Been Tested?

Reproduced and verified on a self-hosted production instance:

1. Real customer reply via email with a PNG screenshot of an in-app
error.

   Before:
   ```ruby
   a = Message.find(<id>).attachments.first
   a.file_type                # => "file"
   a.file.blob.content_type   # => "image/png"
Captain::OpenAiMessageBuilderService.new(message:
a.message).generate_content
   # => [{type: 'text', text: '...'},
# {type: 'text', text: 'User has shared an attachment'}]  no image_url
   ```
Captain reply: "Please copy and paste the full error text…" (model never
saw the image).

2. After the patch + force-recreate, same conversation:
   ```ruby
   a.file_type                # => "image"
Captain::OpenAiMessageBuilderService.new(message:
a.message).generate_content
   # => [{type: 'text',      text: '...'},
# {type: 'image_url', image_url: {url: 'https://.../<blob>.png'}}] 
   ```
Captain reply now correctly references the on-screen error text from the
screenshot via the multimodal vision path — no more deflection.

3. Regression sanity-check on non-image attachments (PDF / Office docs):
`file_type` falls through to `:file`, behavior unchanged.

## Notes for self-hosted operators

Existing email image attachments in the DB will still have `file_type:
'file'`. A one-shot backfill is straightforward and safe (no data loss,
only metadata):

```ruby
Attachment.joins(message: :conversation)
          .where(messages: { content_type: 'incoming_email' })
          .where(file_type: 'file')
          .find_each do |a|
  next unless a.file.attached?
  ct = a.file.blob.content_type.to_s
  next unless ct.start_with?('image/', 'audio/', 'video/')
  new_type = ct.start_with?('image/') ? :image : (ct.start_with?('video/') ? :video : :audio)
  a.update_columns(file_type: Attachment.file_types[new_type])
end
```

## Checklist

- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective — happy to add a
`mailbox_helper_spec` example for `process_regular_attachments` if
maintainers prefer; existing specs in that file focus on inline-image
handling.

---------

Co-authored-by: Aakash Bakhle <48802744+aakashb95@users.noreply.github.com>
2026-05-19 10:11:32 +05:30

126 lines
4.2 KiB
Ruby

module MailboxHelper
include MailboxInlineAttachmentHelper
include ::FileTypeHelper
private
def create_message
Rails.logger.info "[MailboxHelper] Creating message #{processed_mail.message_id}"
return if @conversation.messages.find_by(source_id: processed_mail.message_id).present?
@message = @conversation.messages.create!(
account_id: @conversation.account_id,
sender: @conversation.contact,
content: mail_content&.truncate(150_000),
inbox_id: @conversation.inbox_id,
message_type: 'incoming',
content_type: 'incoming_email',
source_id: processed_mail.message_id,
content_attributes: {
email: processed_mail.serialized_data,
cc_email: processed_mail.cc,
bcc_email: processed_mail.bcc
}
)
end
def add_attachments_to_message
return if @message.blank?
# Load email content once for all attachment processing
load_email_content
# ensure we don't add more than the permitted number of attachments
all_attachments = processed_mail.attachments.last(Message::NUMBER_OF_PERMITTED_ATTACHMENTS)
grouped_attachments = group_attachments(all_attachments)
process_inline_attachments(grouped_attachments[:inline]) if grouped_attachments[:inline].present?
process_regular_attachments(grouped_attachments[:regular]) if grouped_attachments[:regular].present?
@message.save!
end
def group_attachments(attachments)
# If the email lacks a text body or if inline attachments aren't images,
# treat them as standard attachments for processing.
inline_attachments = attachments.select do |attachment|
inline_attachment?(attachment)
end
regular_attachments = attachments - inline_attachments
{ inline: inline_attachments, regular: regular_attachments }
end
def process_regular_attachments(attachments)
Rails.logger.info "[MailboxHelper] Processing regular attachments for message with ID: #{processed_mail.message_id}"
attachments.each do |mail_attachment|
blob = mail_attachment[:blob]
attachment = @message.attachments.new(
account_id: @conversation.account_id,
file_type: file_type(blob.content_type)
)
attachment.file.attach(blob)
end
end
def process_inline_attachments(attachments)
Rails.logger.info "[MailboxHelper] Processing inline attachments for message with ID: #{processed_mail.message_id}"
attachments.each do |mail_attachment|
embed_inline_image_source(mail_attachment)
end
# update the message content with the updated html and text content
@message.content_attributes[:email][:html_content][:full] = @html_content
@message.content_attributes[:email][:text_content][:full] = @text_content
end
def embed_inline_image_source(mail_attachment)
if @html_content.present?
upload_inline_image(mail_attachment)
elsif @text_content.present?
embed_plain_text_email_with_inline_image(mail_attachment)
end
end
def embed_plain_text_email_with_inline_image(mail_attachment)
attachment_name = mail_attachment[:original].filename
img_tag = "<img src=\"#{inline_image_url(mail_attachment[:blob])}\" alt=\"#{attachment_name}\">"
tag_to_replace = "[image: #{attachment_name}]"
if @text_content.include?(tag_to_replace)
@text_content = @text_content.gsub(tag_to_replace, img_tag)
else
@text_content += "\n\n#{img_tag}"
end
end
def inline_image_url(blob)
Rails.application.routes.url_helpers.url_for(blob)
end
def create_contact
@contact_inbox = ::ContactInboxWithContactBuilder.new(
source_id: processed_mail.original_sender,
inbox: @inbox,
contact_attributes: {
name: identify_contact_name,
email: processed_mail.original_sender,
additional_attributes: { source_id: "email:#{processed_mail.message_id}" }
}
).perform
@contact = @contact_inbox.contact
Rails.logger.info "[MailboxHelper] Contact created with ID: #{@contact.id} for inbox with ID: #{@inbox.id}"
end
def mail_content
if processed_mail.text_content.present?
processed_mail.text_content[:reply]
elsif processed_mail.html_content.present?
processed_mail.html_content[:reply]
end
end
end