Files
37eed5de1e feat(whatsapp): Add support for voice messages (#14606)
> Reopened from #13613, now from a personal fork
(`gabrieljablonski/chatwoot`) so maintainers can push edits —
organization-owned forks don't support "Allow edits from maintainers".
The previous PR is closed in favor of this one; same commits, same diff.

## Description

This PR adds support for sending voice messages (voice notes) through
the WhatsApp Cloud API. When agents record audio in Chatwoot, it is now
transcoded in the browser from WebM/Opus to OGG/Opus and sent with the
`voice: true` flag, so it appears as a native voice note bubble on
WhatsApp — not as a file/document attachment.

Closes #13283

**Key Changes:**
- Added `webmOpusToOgg.js` — a pure JS EBML parser + OGG page builder
that remuxes browser-recorded WebM/Opus audio into OGG/Opus entirely
client-side, with no server-side dependencies.
- Updated `AudioRecorder.vue` to use an explicit `mimeType` hint, proper
resource cleanup, and an `AUDIO_EXTENSION_MAP` for correct file
extensions.
- Renamed `mp3ConversionUtils.js` → `audioConversionUtils.js` and added
OGG conversion support via the new remuxer.
- Updated `ReplyBox.vue` to request OGG format for WhatsApp channels,
pass `isVoiceMessage` per-attachment, and handle recording errors with a
user-facing alert.
- Updated `MessageBuilder` to read the `is_voice_message` param and
persist it in attachment metadata.
- Updated `WhatsappCloudService` to:
- Normalize `audio/opus` → `audio/ogg` content type on ActiveStorage
blobs (works around Marcel gem re-detection).
- Send the `voice: true` flag when the attachment is a voice message
with `audio/ogg` content type.
  - Use WhatsApp Cloud API `v24.0` for the attachment endpoint.
- Added `AUDIO_CONVERSION_FAILED` i18n key.

**How it works:**
1. The browser records audio as WebM/Opus (Chrome/Firefox default).
2. `audioConversionUtils.js` remuxes it to OGG/Opus using the pure-JS
`webmOpusToOgg` remuxer — no server transcoding needed.
3. The OGG file is uploaded with `is_voice_message: true` in the form
payload.
4. `MessageBuilder` persists `is_voice_message` in the attachment's
`meta` hash.
5. `WhatsappCloudService` normalizes the blob content type if needed,
then sends the attachment with `voice: true` so WhatsApp renders it as a
voice note.

## Type of change

- [X] New feature (non-breaking change which adds functionality)

## How Has This Been Tested?

1. Record a voice message in a WhatsApp Cloud conversation.
2. Verify the audio is transcoded to OGG (check file extension in the
attachment preview).
3. Verify the message arrives on WhatsApp as a voice note bubble (not a
document/file).
4. Send an image or document attachment and verify it still works as
before (no `voice` flag).
5. Send a regular (non-voice) audio file and verify it arrives without
the voice flag.

---------

Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
Co-authored-by: Muhsin <12408980+muhsin-k@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-02 15:33:32 +04:00

238 lines
7.5 KiB
Ruby

class Messages::MessageBuilder
include ::FileTypeHelper
include ::EmailHelper
include ::DataHelper
attr_reader :message
def initialize(user, conversation, params)
@params = params
@private = params[:private] || false
@conversation = conversation
@user = user
@account = conversation.account
@message_type = params[:message_type] || 'outgoing'
@attachments = params[:attachments]
@is_voice_message = ActiveModel::Type::Boolean.new.cast(params[:is_voice_message])
@automation_rule = content_attributes&.dig(:automation_rule_id)
return unless params.instance_of?(ActionController::Parameters)
@in_reply_to = content_attributes&.dig(:in_reply_to)
@items = content_attributes&.dig(:items)
end
def perform
@message = @conversation.messages.build(message_params)
process_attachments
process_emails
# When the message has no quoted content, it will just be rendered as a regular message
# The frontend is equipped to handle this case
process_email_content
@message.save!
@message
end
private
# Extracts content attributes from the given params.
# - Converts ActionController::Parameters to a regular hash if needed.
# - Attempts to parse a JSON string if content is a string.
# - Returns an empty hash if content is not present, if there's a parsing error, or if it's an unexpected type.
def content_attributes
params = convert_to_hash(@params)
content_attributes = params.fetch(:content_attributes, {})
return safe_parse_json(content_attributes) if content_attributes.is_a?(String)
return content_attributes if content_attributes.is_a?(Hash)
{}
end
def process_attachments
return if @attachments.blank?
@attachments.each do |uploaded_attachment|
attachment = @message.attachments.build(
account_id: @message.account_id,
file: uploaded_attachment
)
attachment.file_type = attachment_file_type(uploaded_attachment)
tag_voice_message(attachment)
end
end
def attachment_file_type(uploaded_attachment)
if uploaded_attachment.is_a?(String)
file_type_by_signed_id(uploaded_attachment)
else
file_type(uploaded_attachment&.content_type)
end
end
def tag_voice_message(attachment)
return unless @is_voice_message && attachment.file_type == 'audio'
attachment.meta = (attachment.meta || {}).merge('is_voice_message' => true)
end
def process_emails
return unless @conversation.inbox&.inbox_type == 'Email'
cc_emails = process_email_string(@params[:cc_emails])
bcc_emails = process_email_string(@params[:bcc_emails])
to_emails = process_email_string(@params[:to_emails])
all_email_addresses = cc_emails + bcc_emails + to_emails
validate_email_addresses(all_email_addresses)
@message.content_attributes[:cc_emails] = cc_emails
@message.content_attributes[:bcc_emails] = bcc_emails
@message.content_attributes[:to_emails] = to_emails
end
def process_email_content
return unless should_process_email_content?
@message.content_attributes ||= {}
email_attributes = build_email_attributes
@message.content_attributes[:email] = email_attributes
end
def process_email_string(email_string)
return [] if email_string.blank?
email_string.gsub(/\s+/, '').split(',')
end
def message_type
if @conversation.inbox.channel_type != 'Channel::Api' && @message_type == 'incoming'
raise StandardError, 'Incoming messages are only allowed in Api inboxes'
end
@message_type
end
def sender
message_type == 'outgoing' ? (message_sender || @user) : @conversation.contact
end
def external_created_at
@params[:external_created_at].present? ? { external_created_at: @params[:external_created_at] } : {}
end
def automation_rule_id
@automation_rule.present? ? { content_attributes: { automation_rule_id: @automation_rule } } : {}
end
def campaign_id
@params[:campaign_id].present? ? { additional_attributes: { campaign_id: @params[:campaign_id] } } : {}
end
def template_params
@params[:template_params].present? ? { additional_attributes: { template_params: JSON.parse(@params[:template_params].to_json) } } : {}
end
def message_sender
return if @params[:sender_type] != 'AgentBot'
AgentBot.where(account_id: [nil, @conversation.account.id]).find_by(id: @params[:sender_id])
end
def message_params
{
account_id: @conversation.account_id,
inbox_id: @conversation.inbox_id,
message_type: message_type,
content: @params[:content],
private: @private,
sender: sender,
content_type: @params[:content_type],
content_attributes: content_attributes.presence,
items: @items,
in_reply_to: @in_reply_to,
echo_id: @params[:echo_id],
source_id: @params[:source_id]
}.merge(external_created_at).merge(automation_rule_id).merge(campaign_id).merge(template_params)
end
def email_inbox?
@conversation.inbox&.inbox_type == 'Email'
end
def should_process_email_content?
email_inbox? && !@private && @message.content.present?
end
def build_email_attributes
email_attributes = ensure_indifferent_access(@message.content_attributes[:email] || {})
normalized_content = normalize_email_body(@message.content)
# Process liquid templates in normalized content with code block protection
processed_content = process_liquid_in_email_body(normalized_content)
# Use custom HTML content if provided, otherwise generate from message content
email_attributes[:html_content] = if custom_email_content_provided?
build_custom_html_content
else
build_html_content(processed_content)
end
email_attributes[:text_content] = build_text_content(processed_content)
email_attributes
end
def build_html_content(normalized_content)
html_content = ensure_indifferent_access(@message.content_attributes.dig(:email, :html_content) || {})
rendered_html = render_email_html(normalized_content)
html_content[:full] = rendered_html
html_content[:reply] = rendered_html
html_content
end
def build_text_content(normalized_content)
text_content = ensure_indifferent_access(@message.content_attributes.dig(:email, :text_content) || {})
text_content[:full] = normalized_content
text_content[:reply] = normalized_content
text_content
end
def custom_email_content_provided?
@params[:email_html_content].present?
end
def build_custom_html_content
html_content = ensure_indifferent_access(@message.content_attributes.dig(:email, :html_content) || {})
html_content[:full] = @params[:email_html_content]
html_content[:reply] = @params[:email_html_content]
html_content
end
# Liquid processing methods for email content
def process_liquid_in_email_body(content)
return content if content.blank?
return content unless should_process_liquid?
# Protect code blocks from liquid processing
modified_content = modified_liquid_content(content)
template = Liquid::Template.parse(modified_content)
template.render(drops_with_sender)
rescue Liquid::Error
content
end
def should_process_liquid?
@message_type == 'outgoing' || @message_type == 'template'
end
def drops_with_sender
message_drops(@conversation).merge({
'agent' => UserDrop.new(sender)
})
end
end
Messages::MessageBuilder.prepend_mod_with('Messages::MessageBuilder')