> Reopened from #13613, now from a personal fork (`gabrieljablonski/chatwoot`) so maintainers can push edits — organization-owned forks don't support "Allow edits from maintainers". The previous PR is closed in favor of this one; same commits, same diff. ## Description This PR adds support for sending voice messages (voice notes) through the WhatsApp Cloud API. When agents record audio in Chatwoot, it is now transcoded in the browser from WebM/Opus to OGG/Opus and sent with the `voice: true` flag, so it appears as a native voice note bubble on WhatsApp — not as a file/document attachment. Closes #13283 **Key Changes:** - Added `webmOpusToOgg.js` — a pure JS EBML parser + OGG page builder that remuxes browser-recorded WebM/Opus audio into OGG/Opus entirely client-side, with no server-side dependencies. - Updated `AudioRecorder.vue` to use an explicit `mimeType` hint, proper resource cleanup, and an `AUDIO_EXTENSION_MAP` for correct file extensions. - Renamed `mp3ConversionUtils.js` → `audioConversionUtils.js` and added OGG conversion support via the new remuxer. - Updated `ReplyBox.vue` to request OGG format for WhatsApp channels, pass `isVoiceMessage` per-attachment, and handle recording errors with a user-facing alert. - Updated `MessageBuilder` to read the `is_voice_message` param and persist it in attachment metadata. - Updated `WhatsappCloudService` to: - Normalize `audio/opus` → `audio/ogg` content type on ActiveStorage blobs (works around Marcel gem re-detection). - Send the `voice: true` flag when the attachment is a voice message with `audio/ogg` content type. - Use WhatsApp Cloud API `v24.0` for the attachment endpoint. - Added `AUDIO_CONVERSION_FAILED` i18n key. **How it works:** 1. The browser records audio as WebM/Opus (Chrome/Firefox default). 2. `audioConversionUtils.js` remuxes it to OGG/Opus using the pure-JS `webmOpusToOgg` remuxer — no server transcoding needed. 3. The OGG file is uploaded with `is_voice_message: true` in the form payload. 4. `MessageBuilder` persists `is_voice_message` in the attachment's `meta` hash. 5. `WhatsappCloudService` normalizes the blob content type if needed, then sends the attachment with `voice: true` so WhatsApp renders it as a voice note. ## Type of change - [X] New feature (non-breaking change which adds functionality) ## How Has This Been Tested? 1. Record a voice message in a WhatsApp Cloud conversation. 2. Verify the audio is transcoded to OGG (check file extension in the attachment preview). 3. Verify the message arrives on WhatsApp as a voice note bubble (not a document/file). 4. Send an image or document attachment and verify it still works as before (no `voice` flag). 5. Send a regular (non-voice) audio file and verify it arrives without the voice flag. --------- Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com> Co-authored-by: Muhsin <12408980+muhsin-k@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
238 lines
7.5 KiB
Ruby
238 lines
7.5 KiB
Ruby
class Messages::MessageBuilder
|
|
include ::FileTypeHelper
|
|
include ::EmailHelper
|
|
include ::DataHelper
|
|
|
|
attr_reader :message
|
|
|
|
def initialize(user, conversation, params)
|
|
@params = params
|
|
@private = params[:private] || false
|
|
@conversation = conversation
|
|
@user = user
|
|
@account = conversation.account
|
|
@message_type = params[:message_type] || 'outgoing'
|
|
@attachments = params[:attachments]
|
|
@is_voice_message = ActiveModel::Type::Boolean.new.cast(params[:is_voice_message])
|
|
@automation_rule = content_attributes&.dig(:automation_rule_id)
|
|
return unless params.instance_of?(ActionController::Parameters)
|
|
|
|
@in_reply_to = content_attributes&.dig(:in_reply_to)
|
|
@items = content_attributes&.dig(:items)
|
|
end
|
|
|
|
def perform
|
|
@message = @conversation.messages.build(message_params)
|
|
process_attachments
|
|
process_emails
|
|
# When the message has no quoted content, it will just be rendered as a regular message
|
|
# The frontend is equipped to handle this case
|
|
process_email_content
|
|
@message.save!
|
|
@message
|
|
end
|
|
|
|
private
|
|
|
|
# Extracts content attributes from the given params.
|
|
# - Converts ActionController::Parameters to a regular hash if needed.
|
|
# - Attempts to parse a JSON string if content is a string.
|
|
# - Returns an empty hash if content is not present, if there's a parsing error, or if it's an unexpected type.
|
|
def content_attributes
|
|
params = convert_to_hash(@params)
|
|
content_attributes = params.fetch(:content_attributes, {})
|
|
|
|
return safe_parse_json(content_attributes) if content_attributes.is_a?(String)
|
|
return content_attributes if content_attributes.is_a?(Hash)
|
|
|
|
{}
|
|
end
|
|
|
|
def process_attachments
|
|
return if @attachments.blank?
|
|
|
|
@attachments.each do |uploaded_attachment|
|
|
attachment = @message.attachments.build(
|
|
account_id: @message.account_id,
|
|
file: uploaded_attachment
|
|
)
|
|
|
|
attachment.file_type = attachment_file_type(uploaded_attachment)
|
|
tag_voice_message(attachment)
|
|
end
|
|
end
|
|
|
|
def attachment_file_type(uploaded_attachment)
|
|
if uploaded_attachment.is_a?(String)
|
|
file_type_by_signed_id(uploaded_attachment)
|
|
else
|
|
file_type(uploaded_attachment&.content_type)
|
|
end
|
|
end
|
|
|
|
def tag_voice_message(attachment)
|
|
return unless @is_voice_message && attachment.file_type == 'audio'
|
|
|
|
attachment.meta = (attachment.meta || {}).merge('is_voice_message' => true)
|
|
end
|
|
|
|
def process_emails
|
|
return unless @conversation.inbox&.inbox_type == 'Email'
|
|
|
|
cc_emails = process_email_string(@params[:cc_emails])
|
|
bcc_emails = process_email_string(@params[:bcc_emails])
|
|
to_emails = process_email_string(@params[:to_emails])
|
|
|
|
all_email_addresses = cc_emails + bcc_emails + to_emails
|
|
validate_email_addresses(all_email_addresses)
|
|
|
|
@message.content_attributes[:cc_emails] = cc_emails
|
|
@message.content_attributes[:bcc_emails] = bcc_emails
|
|
@message.content_attributes[:to_emails] = to_emails
|
|
end
|
|
|
|
def process_email_content
|
|
return unless should_process_email_content?
|
|
|
|
@message.content_attributes ||= {}
|
|
email_attributes = build_email_attributes
|
|
@message.content_attributes[:email] = email_attributes
|
|
end
|
|
|
|
def process_email_string(email_string)
|
|
return [] if email_string.blank?
|
|
|
|
email_string.gsub(/\s+/, '').split(',')
|
|
end
|
|
|
|
def message_type
|
|
if @conversation.inbox.channel_type != 'Channel::Api' && @message_type == 'incoming'
|
|
raise StandardError, 'Incoming messages are only allowed in Api inboxes'
|
|
end
|
|
|
|
@message_type
|
|
end
|
|
|
|
def sender
|
|
message_type == 'outgoing' ? (message_sender || @user) : @conversation.contact
|
|
end
|
|
|
|
def external_created_at
|
|
@params[:external_created_at].present? ? { external_created_at: @params[:external_created_at] } : {}
|
|
end
|
|
|
|
def automation_rule_id
|
|
@automation_rule.present? ? { content_attributes: { automation_rule_id: @automation_rule } } : {}
|
|
end
|
|
|
|
def campaign_id
|
|
@params[:campaign_id].present? ? { additional_attributes: { campaign_id: @params[:campaign_id] } } : {}
|
|
end
|
|
|
|
def template_params
|
|
@params[:template_params].present? ? { additional_attributes: { template_params: JSON.parse(@params[:template_params].to_json) } } : {}
|
|
end
|
|
|
|
def message_sender
|
|
return if @params[:sender_type] != 'AgentBot'
|
|
|
|
AgentBot.where(account_id: [nil, @conversation.account.id]).find_by(id: @params[:sender_id])
|
|
end
|
|
|
|
def message_params
|
|
{
|
|
account_id: @conversation.account_id,
|
|
inbox_id: @conversation.inbox_id,
|
|
message_type: message_type,
|
|
content: @params[:content],
|
|
private: @private,
|
|
sender: sender,
|
|
content_type: @params[:content_type],
|
|
content_attributes: content_attributes.presence,
|
|
items: @items,
|
|
in_reply_to: @in_reply_to,
|
|
echo_id: @params[:echo_id],
|
|
source_id: @params[:source_id]
|
|
}.merge(external_created_at).merge(automation_rule_id).merge(campaign_id).merge(template_params)
|
|
end
|
|
|
|
def email_inbox?
|
|
@conversation.inbox&.inbox_type == 'Email'
|
|
end
|
|
|
|
def should_process_email_content?
|
|
email_inbox? && !@private && @message.content.present?
|
|
end
|
|
|
|
def build_email_attributes
|
|
email_attributes = ensure_indifferent_access(@message.content_attributes[:email] || {})
|
|
normalized_content = normalize_email_body(@message.content)
|
|
|
|
# Process liquid templates in normalized content with code block protection
|
|
processed_content = process_liquid_in_email_body(normalized_content)
|
|
|
|
# Use custom HTML content if provided, otherwise generate from message content
|
|
email_attributes[:html_content] = if custom_email_content_provided?
|
|
build_custom_html_content
|
|
else
|
|
build_html_content(processed_content)
|
|
end
|
|
|
|
email_attributes[:text_content] = build_text_content(processed_content)
|
|
email_attributes
|
|
end
|
|
|
|
def build_html_content(normalized_content)
|
|
html_content = ensure_indifferent_access(@message.content_attributes.dig(:email, :html_content) || {})
|
|
rendered_html = render_email_html(normalized_content)
|
|
html_content[:full] = rendered_html
|
|
html_content[:reply] = rendered_html
|
|
html_content
|
|
end
|
|
|
|
def build_text_content(normalized_content)
|
|
text_content = ensure_indifferent_access(@message.content_attributes.dig(:email, :text_content) || {})
|
|
text_content[:full] = normalized_content
|
|
text_content[:reply] = normalized_content
|
|
text_content
|
|
end
|
|
|
|
def custom_email_content_provided?
|
|
@params[:email_html_content].present?
|
|
end
|
|
|
|
def build_custom_html_content
|
|
html_content = ensure_indifferent_access(@message.content_attributes.dig(:email, :html_content) || {})
|
|
|
|
html_content[:full] = @params[:email_html_content]
|
|
html_content[:reply] = @params[:email_html_content]
|
|
|
|
html_content
|
|
end
|
|
|
|
# Liquid processing methods for email content
|
|
def process_liquid_in_email_body(content)
|
|
return content if content.blank?
|
|
return content unless should_process_liquid?
|
|
|
|
# Protect code blocks from liquid processing
|
|
modified_content = modified_liquid_content(content)
|
|
template = Liquid::Template.parse(modified_content)
|
|
template.render(drops_with_sender)
|
|
rescue Liquid::Error
|
|
content
|
|
end
|
|
|
|
def should_process_liquid?
|
|
@message_type == 'outgoing' || @message_type == 'template'
|
|
end
|
|
|
|
def drops_with_sender
|
|
message_drops(@conversation).merge({
|
|
'agent' => UserDrop.new(sender)
|
|
})
|
|
end
|
|
end
|
|
|
|
Messages::MessageBuilder.prepend_mod_with('Messages::MessageBuilder')
|