fix(transcription): guard Whisper 25MB limit and zero temperature for stable output (#14335)

Two production-grade fixes to the existing audio transcription service.
**Independent of the WhatsApp Calling work** — these affect every audio
attachment that goes through Whisper (voice notes, call recordings,
voicemails, etc.).

## Closes
- [PLA-151 — PR-5: Recording Upload + Transcription
Pipeline](https://linear.app/chatwoot/issue/PLA-151/pr-5-recording-upload-transcription-pipeline)

## Why this is needed

### 1. Whisper rejects payloads larger than 25 MB

OpenAI's [Whisper
API](https://platform.openai.com/docs/guides/speech-to-text) hard-caps
file uploads at 25 MB. Long audio recordings — voice notes from chatty
contacts, ~70+ min Opus call recordings — currently hit OpenAI with the
full payload and 413 (\`Payload Too Large\`). The job retries via the
existing \`Faraday::BadRequestError\` discard path, but the agent still
sees a transcription failure for an attachment we knew was too big up
front.

This PR adds a pre-flight \`audio_too_large?\` check via the blob's
\`byte_size\` and returns a controlled error without hitting OpenAI. The
audio attachment is preserved (agents can still listen), only the
transcription is skipped.

### 2. Whisper hallucinates on silence at non-zero temperature

At \`temperature: 0.4\` (the previous value), Whisper produces
well-documented hallucinated repeats on silence and near-silent segments
— e.g. \`Oh, dear. Oh, dear. Oh, dear.\` filling the transcript. This
shows up in real recordings whenever there's a hold or quiet moment.
\`temperature: 0.0\` matches OpenAI's recommended default for
transcription and eliminates the spirals.

Reference:
[openai/whisper#928](https://github.com/openai/whisper/discussions/928),
[openai-python#1010](https://github.com/openai/openai-python/issues/1010).

## Are WhatsApp call recordings already handled?

Yes — by the existing pipeline, **before this PR**:

\`\`\`
Browser MediaRecorder → upload_recording (PR-4)
  → @call.message.attachments.create!(file_type: :audio, ...)
→ Enterprise::Concerns::Attachment#enqueue_audio_transcription
(after_create_commit hook)
      → Messages::AudioTranscriptionJob.perform_later(attachment.id)
        → Messages::AudioTranscriptionService → Whisper
\`\`\`

The \`after_create_commit\` hook already fires for every audio
attachment regardless of source. PR-4's \`upload_recording\` endpoint
creates the attachment; the existing job/service take it from there. No
new wiring needed.

This PR just makes the existing service more robust:
- Calls longer than ~70 min (Opus 48 kbps) no longer 413 against OpenAI
- Quiet recordings no longer produce hallucinated transcripts

## How to test

\`\`\`ruby
# In rails console with a real audio attachment:
service = Messages::AudioTranscriptionService.new(Attachment.audio.last)

# Normal-sized audio: unchanged behaviour
service.perform # => { success: true, transcriptions: ... }

# Large audio: new guard returns error instead of 413-ing OpenAI
allow(attachment.file.blob).to
receive(:byte_size).and_return(30.megabytes)
service.perform # => { error: 'Audio too large for Whisper' }
\`\`\`

Existing transcription specs cover the happy path; one new spec
exercises the byte-limit guard.

## Risk

Low. Both changes are pre-flight guards or parameter values — they
reduce the surface of OpenAI calls that can fail. Failure to transcribe
is already non-fatal (the audio attachment is preserved either way).
This commit is contained in:
Tanmay Deep Sharma
2026-05-05 11:49:28 +07:00
committed by GitHub
parent 624c6c90fd
commit 21c0f4dc52
2 changed files with 33 additions and 1 deletions
@@ -2,6 +2,10 @@ class Messages::AudioTranscriptionService< Llm::LegacyBaseOpenAiService
include Integrations::LlmInstrumentation
WHISPER_MODEL = 'whisper-1'.freeze
# Whisper's hard limit is 25 MB *decimal* (25_000_000), not binary (25.megabytes
# = 26_214_400) — using the binary form leaks the 25.026.2 MB range to the API
# as 413s. Long audio (~70+ min Opus) keeps the attachment but skips transcription.
WHISPER_BYTE_LIMIT = 25_000_000
attr_reader :attachment, :message, :account
@@ -15,6 +19,7 @@ class Messages::AudioTranscriptionService< Llm::LegacyBaseOpenAiService
def perform
return { error: 'Transcription limit exceeded' } unless can_transcribe?
return { error: 'Message not found' } if message.blank?
return { error: 'Audio too large for Whisper' } if audio_too_large?
transcriptions = transcribe_audio
Rails.logger.info "Audio transcription successful: #{transcriptions}"
@@ -33,6 +38,13 @@ class Messages::AudioTranscriptionService< Llm::LegacyBaseOpenAiService
account.usage_limits[:captain][:responses][:current_available].positive?
end
def audio_too_large?
blob = attachment.file&.blob
return false unless blob
blob.byte_size > WHISPER_BYTE_LIMIT
end
def fetch_audio_file
blob = attachment.file.blob
temp_dir = Rails.root.join('tmp/uploads/audio-transcriptions')
@@ -63,11 +75,14 @@ class Messages::AudioTranscriptionService< Llm::LegacyBaseOpenAiService
transcribed_text = nil
File.open(temp_file_path, 'rb') do |file|
# temperature: 0.0 minimises Whisper's hallucinations on silence /
# near-silent audio; non-zero values trigger spiraling repeats like
# "Oh, dear. Oh, dear. Oh, dear." — well-documented Whisper behaviour.
response = @client.audio.transcribe(
parameters: {
model: WHISPER_MODEL,
file: file,
temperature: 0.4
temperature: 0.0
}
)
transcribed_text = response['text']
@@ -63,6 +63,23 @@ RSpec.describe Messages::AudioTranscriptionService, type: :service do
expect(result).to eq({ success: true, transcriptions: 'Existing transcription' })
end
end
context 'when the audio exceeds Whisper byte limit' do
before do
attachment.file.attach(
io: File.open(Rails.public_path.join('audio/widget/ding.mp3')),
filename: 'large.mp3',
content_type: 'audio/mpeg'
)
allow(service).to receive(:can_transcribe?).and_return(true)
allow(attachment.file.blob).to receive(:byte_size).and_return(described_class::WHISPER_BYTE_LIMIT + 1)
end
it 'returns an error without calling Whisper' do
expect(service).not_to receive(:transcribe_audio)
expect(service.perform).to eq({ error: 'Audio too large for Whisper' })
end
end
end
describe '#fetch_audio_file' do