fix: serialize multimodal Captain session content (#15096)
Captain now saves agent session records when a user message includes an image. The saved record keeps the image URL and excludes downloaded image bytes, so image replies no longer report a JSON serialization error after delivery. Fixes: https://chatwoot-p3.sentry.io/issues/7618423184/?alert_rule_id=13673680&alert_type=issue¬ification_uuid=d22a7ab9-95d6-4bba-85e0-733a28466775&project=6382945 ## Root cause RubyLLM downloads image attachments and caches the binary bytes inside `RubyLLM::Content`. `SessionCaptureService` passed the live object to the `run_context` JSON column. Rails then tried to encode the cached JPEG bytes as UTF-8 and raised `JSON::GeneratorError`. The error did not block replies, handoffs, or credit updates because session capture rescues its own failures. The failed write meant that Chatwoot lost the agent session record for the response. ## How to reproduce 1. Send an image to a Captain V2 assistant. 2. Let RubyLLM load the image during the model request. 3. Save the resulting conversation history in an agent session. 4. Observe the JSON encoding error when Rails reaches the cached image bytes. ## What changed `SessionCaptureService` now converts `RubyLLM::Content` to its JSON safe hash before saving the current turn. The hash contains the message text and attachment URL without the cached bytes. Other message content is unchanged. The focused service spec covers a cached JPEG byte payload and passes with 12 examples. RuboCop reports no offenses in the changed service and spec.
This commit is contained in:
@@ -59,6 +59,11 @@ class Captain::Assistant::SessionCaptureService
|
||||
def current_turn_history
|
||||
history = Array(context[:conversation_history])
|
||||
last_user_index = history.rindex { |message| message[:role].to_s == 'user' }
|
||||
last_user_index ? history[last_user_index..] : history
|
||||
current_turn = last_user_index ? history[last_user_index..] : history
|
||||
|
||||
current_turn.map do |message|
|
||||
content = message[:content]
|
||||
content.is_a?(RubyLLM::Content) ? message.merge(content: content.to_h) : message
|
||||
end
|
||||
end
|
||||
end
|
||||
|
||||
@@ -111,6 +111,25 @@ RSpec.describe Captain::Assistant::SessionCaptureService do
|
||||
expect(history.first).to include('role' => 'user', 'content' => 'CUST001')
|
||||
end
|
||||
|
||||
it 'stores multimodal content without cached attachment bytes' do
|
||||
content = RubyLLM::Content.new('See image', ['https://example.com/image.jpg'])
|
||||
content.attachments.first.instance_variable_set(:@content, "\xFF\xD8\xFF\xE0JFIF".b)
|
||||
run_context[:conversation_history] = [
|
||||
{ role: :user, content: content },
|
||||
{ role: :assistant, content: 'I can see the image', agent_name: 'Assistant' }
|
||||
]
|
||||
|
||||
history = service.capture!.run_context
|
||||
|
||||
expect(history.first).to include(
|
||||
'role' => 'user',
|
||||
'content' => {
|
||||
'text' => 'See image',
|
||||
'attachments' => [{ 'type' => 'image', 'source' => 'https://example.com/image.jpg' }]
|
||||
}
|
||||
)
|
||||
end
|
||||
|
||||
it 'stores the full history when it contains no user message' do
|
||||
run_context[:conversation_history] = conversation_history.reject { |message| message[:role] == :user }
|
||||
|
||||
|
||||
Reference in New Issue
Block a user