Commit Graph
6 Commits
Author SHA1 Message Date
Tanmay Deep SharmaandGitHub 68e358d732 feat: voice-call UX fixes (#14579)
## Linear ticket
https://linear.app/chatwoot/issue/CW-7187/voice-calls-followup-tasks

## Description

Improvements to the WhatsApp voice-calling experience plus a cheaper,
more accurate audio-transcription model.
- First-time callers now get a real name. An inbound WhatsApp call
creates the contact from the caller's WhatsApp profile name instead of
the bare phone number.
- Clear, consistent call attribution. Call bubbles show a unified
"Handled by {agent}"
- Cleaner call widget. The dismiss (✕) button is shown only for incoming
calls
- WhatsApp calling for manual inboxes. voice_calling_supported? now
covers any whatsapp_cloud inbox
- Transcription: whisper-1 → gpt-4o-mini-transcribe.

## Type of change

- [ ] New feature (non-breaking change which adds functionality)


## Checklist:

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged and published in downstream
modules
2026-05-27 16:16:16 +05:30
Tanmay Deep SharmaandGitHub 21c0f4dc52 fix(transcription): guard Whisper 25MB limit and zero temperature for stable output (#14335)
Two production-grade fixes to the existing audio transcription service.
**Independent of the WhatsApp Calling work** — these affect every audio
attachment that goes through Whisper (voice notes, call recordings,
voicemails, etc.).

## Closes
- [PLA-151 — PR-5: Recording Upload + Transcription
Pipeline](https://linear.app/chatwoot/issue/PLA-151/pr-5-recording-upload-transcription-pipeline)

## Why this is needed

### 1. Whisper rejects payloads larger than 25 MB

OpenAI's [Whisper
API](https://platform.openai.com/docs/guides/speech-to-text) hard-caps
file uploads at 25 MB. Long audio recordings — voice notes from chatty
contacts, ~70+ min Opus call recordings — currently hit OpenAI with the
full payload and 413 (\`Payload Too Large\`). The job retries via the
existing \`Faraday::BadRequestError\` discard path, but the agent still
sees a transcription failure for an attachment we knew was too big up
front.

This PR adds a pre-flight \`audio_too_large?\` check via the blob's
\`byte_size\` and returns a controlled error without hitting OpenAI. The
audio attachment is preserved (agents can still listen), only the
transcription is skipped.

### 2. Whisper hallucinates on silence at non-zero temperature

At \`temperature: 0.4\` (the previous value), Whisper produces
well-documented hallucinated repeats on silence and near-silent segments
— e.g. \`Oh, dear. Oh, dear. Oh, dear.\` filling the transcript. This
shows up in real recordings whenever there's a hold or quiet moment.
\`temperature: 0.0\` matches OpenAI's recommended default for
transcription and eliminates the spirals.

Reference:
[openai/whisper#928](https://github.com/openai/whisper/discussions/928),
[openai-python#1010](https://github.com/openai/openai-python/issues/1010).

## Are WhatsApp call recordings already handled?

Yes — by the existing pipeline, **before this PR**:

\`\`\`
Browser MediaRecorder → upload_recording (PR-4)
  → @call.message.attachments.create!(file_type: :audio, ...)
→ Enterprise::Concerns::Attachment#enqueue_audio_transcription
(after_create_commit hook)
      → Messages::AudioTranscriptionJob.perform_later(attachment.id)
        → Messages::AudioTranscriptionService → Whisper
\`\`\`

The \`after_create_commit\` hook already fires for every audio
attachment regardless of source. PR-4's \`upload_recording\` endpoint
creates the attachment; the existing job/service take it from there. No
new wiring needed.

This PR just makes the existing service more robust:
- Calls longer than ~70 min (Opus 48 kbps) no longer 413 against OpenAI
- Quiet recordings no longer produce hallucinated transcripts

## How to test

\`\`\`ruby
# In rails console with a real audio attachment:
service = Messages::AudioTranscriptionService.new(Attachment.audio.last)

# Normal-sized audio: unchanged behaviour
service.perform # => { success: true, transcriptions: ... }

# Large audio: new guard returns error instead of 413-ing OpenAI
allow(attachment.file.blob).to
receive(:byte_size).and_return(30.megabytes)
service.perform # => { error: 'Audio too large for Whisper' }
\`\`\`

Existing transcription specs cover the happy path; one new spec
exercises the byte-limit guard.

## Risk

Low. Both changes are pre-flight guards or parameter values — they
reduce the surface of OpenAI calls that can fail. Failure to transcribe
is already non-fatal (the audio attachment is preserved either way).
2026-05-05 11:49:28 +07:00
Sojan JoseandGitHub 61eaa098ae fix(messages): reduce audio transcription 400 retry noise (#13487)
## Summary
This PR reduces duplicate failure noise for audio transcription jobs
that fail with permanent HTTP 400 responses, and fixes a file-format
edge case causing intermittent 400s.

Sentry issue: [CHATWOOT-99E /
6660541334](https://chatwoot-p3.sentry.io/issues/6660541334/)

## Confirmed root cause
For some attachments, the stored filename had no extension (example:
`speech`, content type `audio/mpeg`).
When the temporary transcription upload file was created without an
extension, OpenAI returned:
`Unrecognized file format` (HTTP 400).

## Scope of changes
1. `Messages::AudioTranscriptionJob`
- Keeps `discard_on Faraday::BadRequestError` to avoid retry storms on
permanent request errors.
- Adds explicit Rails warning logs for discarded jobs with
attachment/job/status context.

2. `Messages::AudioTranscriptionService`
- Keeps guaranteed temp file cleanup via `ensure`.
- Ensures temp upload files include an extension when the original
filename has none, derived from blob `content_type`.
- This addresses intermittent failures like extensionless `audio/mpeg`
files.

## Reproduction
Enable audio transcription for an account and process an audio
attachment whose stored filename has no extension (for example `speech`)
but valid audio content type (`audio/mpeg`).
Before this fix, OpenAI transcription could return HTTP 400
`Unrecognized file format` for that attachment while similar attachments
with extensions succeeded.

## Testing
Ran:
`bundle exec rubocop
enterprise/app/jobs/messages/audio_transcription_job.rb
enterprise/app/services/messages/audio_transcription_service.rb`

Result: both modified files pass lint with no offenses.
2026-02-17 13:25:13 +05:30
PranavandGitHub ea4477ccde fix: Update ActiveStorage::FileNotFoundError error and fix the captain condition in audio transcription (#11779)
Update the error to `ActiveStorage::FileNotFoundError`. Fix the
condition to enable audio transcription and added a spec for it.
2025-06-20 13:20:55 -07:00
PranavandGitHub 9b43a0f72b fix: Retry job if file not found (#11683)
Removed StandardError rescue blocks and added retry_on for
ResponseBuilderJob and AudioTranscriptionJob
2025-06-05 22:53:11 -05:00
8bc00f707b feat(ee): Add transcription support for audio messages (#11670)
<img width="419" alt="Screenshot 2025-06-03 at 4 25 37 PM"
src="https://github.com/user-attachments/assets/4b6ddd11-9b91-4981-a571-83746cc4d40b"
/>


Fixes https://github.com/chatwoot/chatwoot/issues/10182

---------

Co-authored-by: Sojan Jose <sojan@pepalo.com>
2025-06-05 18:29:37 -05:00