From ae49af354d018f17935f8824921c9c29c9d27948 Mon Sep 17 00:00:00 2001 From: Aakash Bakhle <48802744+aakashb95@users.noreply.github.com> Date: Tue, 21 Jul 2026 13:04:12 +0530 Subject: [PATCH] fix: serialize multimodal Captain session content (#15096) Captain now saves agent session records when a user message includes an image. The saved record keeps the image URL and excludes downloaded image bytes, so image replies no longer report a JSON serialization error after delivery. Fixes: https://chatwoot-p3.sentry.io/issues/7618423184/?alert_rule_id=13673680&alert_type=issue¬ification_uuid=d22a7ab9-95d6-4bba-85e0-733a28466775&project=6382945 ## Root cause RubyLLM downloads image attachments and caches the binary bytes inside `RubyLLM::Content`. `SessionCaptureService` passed the live object to the `run_context` JSON column. Rails then tried to encode the cached JPEG bytes as UTF-8 and raised `JSON::GeneratorError`. The error did not block replies, handoffs, or credit updates because session capture rescues its own failures. The failed write meant that Chatwoot lost the agent session record for the response. ## How to reproduce 1. Send an image to a Captain V2 assistant. 2. Let RubyLLM load the image during the model request. 3. Save the resulting conversation history in an agent session. 4. Observe the JSON encoding error when Rails reaches the cached image bytes. ## What changed `SessionCaptureService` now converts `RubyLLM::Content` to its JSON safe hash before saving the current turn. The hash contains the message text and attachment URL without the cached bytes. Other message content is unchanged. The focused service spec covers a cached JPEG byte payload and passes with 12 examples. RuboCop reports no offenses in the changed service and spec. --- .../assistant/session_capture_service.rb | 7 ++++++- .../assistant/session_capture_service_spec.rb | 19 +++++++++++++++++++ 2 files changed, 25 insertions(+), 1 deletion(-) diff --git a/enterprise/app/services/captain/assistant/session_capture_service.rb b/enterprise/app/services/captain/assistant/session_capture_service.rb index 36af50308..ceeb28210 100644 --- a/enterprise/app/services/captain/assistant/session_capture_service.rb +++ b/enterprise/app/services/captain/assistant/session_capture_service.rb @@ -59,6 +59,11 @@ class Captain::Assistant::SessionCaptureService def current_turn_history history = Array(context[:conversation_history]) last_user_index = history.rindex { |message| message[:role].to_s == 'user' } - last_user_index ? history[last_user_index..] : history + current_turn = last_user_index ? history[last_user_index..] : history + + current_turn.map do |message| + content = message[:content] + content.is_a?(RubyLLM::Content) ? message.merge(content: content.to_h) : message + end end end diff --git a/spec/enterprise/services/captain/assistant/session_capture_service_spec.rb b/spec/enterprise/services/captain/assistant/session_capture_service_spec.rb index ef25b1e2b..daf38c8d6 100644 --- a/spec/enterprise/services/captain/assistant/session_capture_service_spec.rb +++ b/spec/enterprise/services/captain/assistant/session_capture_service_spec.rb @@ -111,6 +111,25 @@ RSpec.describe Captain::Assistant::SessionCaptureService do expect(history.first).to include('role' => 'user', 'content' => 'CUST001') end + it 'stores multimodal content without cached attachment bytes' do + content = RubyLLM::Content.new('See image', ['https://example.com/image.jpg']) + content.attachments.first.instance_variable_set(:@content, "\xFF\xD8\xFF\xE0JFIF".b) + run_context[:conversation_history] = [ + { role: :user, content: content }, + { role: :assistant, content: 'I can see the image', agent_name: 'Assistant' } + ] + + history = service.capture!.run_context + + expect(history.first).to include( + 'role' => 'user', + 'content' => { + 'text' => 'See image', + 'attachments' => [{ 'type' => 'image', 'source' => 'https://example.com/image.jpg' }] + } + ) + end + it 'stores the full history when it contains no user message' do run_context[:conversation_history] = conversation_history.reject { |message| message[:role] == :user }