akadmin/chatwoot
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

chore(whatsapp-call): remove internal planning docs from PR

Browse Source 
Drop the in-repo planning/spec docs that were accumulated while iterating on
the feature. They duplicate one another and belong in an internal wiki, not
shipped with the PR.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Tanmay Deep Sharma
2026-04-21 12:18:40 +07:00
co-authored by Claude Opus 4.7
parent 5e5dc21f2f
commit fef9fdbd56
6 changed files with 0 additions and 5311 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/SERVER_SIDE_WEBRTC_ARCHITECTURE.md
-1504
View File
File diff suppressed because it is too large Load Diff
docs/SERVER_SIDE_WEBRTC_IMPLEMENTATION_PLAN.md
-1314
View File
File diff suppressed because it is too large Load Diff
docs/WHATSAPP_CALL_FEATURE_SPEC.md
-272
View File
@@ -1,272 +0,0 @@
# WhatsApp Calling — Feature Spec
> **Edition:** Enterprise (Premium) | **Feature Flag:** `whatsapp_call` | **Branch:** `feat/whatsapp-call`
## What is it?
Agents can **receive and make voice calls** with WhatsApp contacts directly from the Chatwoot dashboard. Calls are browser-based (WebRTC) — no Twilio, no phone hardware, zero telephony cost.
Calls are **automatically recorded** and **transcribed** (OpenAI Whisper), and appear inline in the conversation thread.
---
## How it works (30-second version)
```
SIGNALING AUDIO
(who calls whom, (actual voice data)
accept/reject)
┌──────────┐ REST / Webhooks ┌──────────────┐
│ WhatsApp │◄──────────────────────►│ Chatwoot │
│ Contact │ │ Backend │
└────┬─────┘ └──────┬───────┘
│ │ ActionCable (WebSocket)
│ │
│ ┌────────────────┐ ┌──────▼───────┐
└─────►│ Meta Media │◄════►│ Agent's │
│ Servers │ SRTP │ Browser │
└────────────────┘ └──────────────┘
Key insight: Chatwoot backend handles signaling only.
Audio flows directly between browser and Meta.
```
---
## Call lifecycle
```
┌─────────┐ ┌──────────┐ ┌────────┐ ┌────────────┐
│ RINGING │────►│ ACCEPTED │────►│ ENDED │────►│ TRANSCRIBED│
└─────────┘ └──────────┘ └────────┘ └────────────┘
│
├──► REJECTED
├──► MISSED (30s timeout)
└──► FAILED
```
---
## Inbound call flow
```
Customer dials business number
│
▼
Meta sends webhook ──► IncomingCallService
├── Find/create contact & conversation
├── Create Call record (status: ringing)
├── Create voice_call message
└── Broadcast via ActionCable
│
▼
All agents see floating widget
with Accept / Reject buttons
│
Agent clicks Accept
├── Browser requests mic (getUserMedia)
├── WebRTC peer connection created
├── SDP answer generated + sent to backend
└── Backend relays to Meta API
│
▼
Audio flows (Meta servers ↔ Browser)
Recording starts automatically
│
Call ends (either side hangs up)
├── Recording uploaded
└── Transcription triggered (async)
```
## Outbound call flow
```
Agent clicks phone icon in conversation header
│
▼
Browser creates WebRTC offer ──► POST /whatsapp_calls/initiate
│
┌──────┴──────┐
│ │
Success Error 138006
│ (no permission)
│ │
│ Send permission request
│ to customer via WhatsApp
│ │
│ Customer approves ──► retry call
│
Customer's phone rings
│
Meta webhook with SDP answer
│
Audio connected + recording starts
```
## Call permission flow
Meta requires customers to **opt-in** before a business can call them. If the customer hasn't opted in:
1. Agent tries to call → Meta returns error `138006`
2. Chatwoot auto-sends an interactive permission request to the customer
3. Customer approves on WhatsApp
4. Agent retries the call → succeeds
5. Rate-limited: 1 permission request per 5 minutes per conversation
---
## UI touchpoints
| Where | What |
|-------|------|
| **Conversation Header** | Phone icon to initiate outbound calls |
| **Floating Widget** (bottom-right) | Incoming call notification (accept/reject), active call controls (mute/hangup/timer) |
| **Message Bubble** (VoiceCall type) | Call status, duration, recording audio player, transcript (expandable), "answered by" agent |
| **Inbox Settings** | Toggle to enable/disable calling per WhatsApp inbox |
---
## Data model
```
calls table
├── account_id, inbox_id, conversation_id
├── message_id, accepted_by_agent_id
├── provider (enum: twilio=0, whatsapp=1)
├── direction (enum: incoming=0, outgoing=1)
├── status (ringing → accepted → ended)
├── duration_seconds, end_reason
├── meta (jsonb: SDP offer/answer, ICE servers)
├── transcript (text)
└── recording (ActiveStorage attachment)
```
Voice calls create messages with `content_type: 'voice_call'` containing call metadata in `content_attributes`.
---
## API endpoints
**Base:** `POST /api/v1/accounts/:account_id/whatsapp_calls`
| Endpoint | What it does |
|----------|-------------|
| `GET /:id` | Get call details (SDP offer, ICE servers, caller info) |
| `POST /:id/accept` | Accept call with SDP answer |
| `POST /:id/reject` | Reject incoming call |
| `POST /:id/terminate` | End active call |
| `POST /initiate` | Start outbound call with SDP offer |
| `POST /:id/upload_recording` | Upload recorded audio blob |
## ActionCable events
| Event | When |
|-------|------|
| `whatsapp_call.incoming` | New inbound call (all agents receive) |
| `whatsapp_call.accepted` | Call accepted (dismisses widget for other agents) |
| `whatsapp_call.ended` | Call terminated |
| `whatsapp_call.outbound_connected` | Outbound call answered by customer |
| `whatsapp_call.permission_granted` | Customer approved call permission |
---
## Recording & transcription
```
Agent mic ──┐
├──► AudioContext mixer ──► MediaRecorder ──► WebM blob
Remote audio┘ │
Upload to server
│
ActiveStorage attachment
│
CallTranscriptionJob (async)
│
OpenAI Whisper API
(whisper-1, temp 0.4)
│
Transcript stored on
call + message bubble
```
**Requires:** `captain_integration` feature flag + OpenAI API key configured.
---
## Enterprise architecture
All calling code lives under `enterprise/`. Integration with OSS via `prepend_mod_with`:
```
enterprise/app/
├── controllers/api/v1/accounts/whatsapp_calls_controller.rb
├── models/call.rb
├── services/whatsapp/
│ ├── call_service.rb (accept/reject/terminate orchestration)
│ ├── incoming_call_service.rb (webhook processing)
│ ├── call_message_builder.rb (voice_call message creation)
│ ├── call_permission_reply_service.rb
│ ├── call_transcription_service.rb
│ └── providers/whatsapp_cloud_call_methods.rb (Meta API HTTP layer)
└── jobs/
├── whatsapp/call_transcription_job.rb
└── enterprise/webhooks/whatsapp_events_job.rb
```
**OSS changes (minimal):** `WhatsappEventsJob` gets a `prepend_mod_with` hook + `ActionCableListener` nil safety fix.
---
## Key technical details
| Topic | Detail |
|-------|--------|
| **SDP fix** | Meta requires `a=setup:active` but WebRTC generates `a=setup:actpass` — rewritten server-side |
| **ICE gathering** | 10s timeout, default STUN: `stun:stun.l.google.com:19302` |
| **Module-scope WebRTC** | `RTCPeerConnection`/`MediaStream` stored outside Pinia (not serializable) |
| **Multi-agent routing** | All agents see incoming call; first to accept wins, others get dismissed |
| **Page unload** | Active calls terminated via `fetch({ keepalive: true })` on `beforeunload` |
| **Recording format** | `audio/webm;codecs=opus`, chunked at 1s intervals |
---
## Limitations
| Constraint | Why |
|------------|-----|
| Browser-only | WebRTC requires a browser — no mobile app or SIP phones |
| No rejoin after refresh | P2P connection (unlike Twilio's conference model) |
| Audio only | Meta API doesn't support video calling |
| One call per agent | Single active call at a time |
| Client-side recording | Lost if browser crashes mid-call |
| No TURN server | May fail behind restrictive firewalls blocking UDP |
| Customer opt-in needed | Outbound calls require Meta's permission flow |
---
## WhatsApp vs Twilio comparison
| | WhatsApp Calling | Twilio Voice |
|---|---|---|
| **Model** | Peer-to-peer (browser ↔ Meta) | Conference (Twilio media server) |
| **Cost** | Free | Per-minute billing |
| **Rejoin** | Not possible | Yes (conference persists) |
| **Recording** | Client-side (browser) | Server-side (Twilio) |
| **Video** | Not supported | Possible |
| **SIP phones** | Not supported | Supported |
---
## How to test
1. Enable `whatsapp_call` feature flag on account
2. Enable calling in WhatsApp Cloud inbox settings
3. Test inbound: call the business WhatsApp number → widget appears → accept → audio flows
4. Test outbound: click phone icon in conversation → customer answers → audio flows
5. Verify: recording uploads, transcript appears in message bubble
6. Verify: multiple agents see incoming call, first-to-accept wins
---
*See [PR Breakdown](./WHATSAPP_CALL_PR_BREAKDOWN.md) for the 9-PR implementation plan.*
docs/WHATSAPP_CALL_PR_BREAKDOWN.md
-347
View File
@@ -1,347 +0,0 @@
# WhatsApp Calling — PR Breakdown Plan
> **Branch:** `feat/whatsapp-call` → **Base:** `develop`
> **Date:** 2026-04-14 (updated)
> **Goal:** Split the monolithic feature branch into **8 backend + 1 frontend = 9 mergeable PRs**, each covering a single concept.
---
## Merge Order & Dependency Graph
```
BACKEND FRONTEND
PR-1: Call Model + Migration ✅ MERGED
│
├── PR-2: Meta API Provider Methods
│ │
│ ├── PR-3: Inbound Webhook Pipeline
│ │ │
│ │ └── PR-4: Call Service +
│ │ Controller + Routes ──────► PR-9: All Frontend Changes
│ │ (API, Store, WebRTC, Widget,
│ └─────────────────────────────────► Bubble, Outbound UI,
│ ActionCable, Settings,
├── PR-5: Transcription Pipeline Voice Guard, i18n, Feature Flag)
│
├── PR-6: Feature Flag + Enterprise Gating
│
├── PR-7: OSS Touchpoints (webhook job refactor + ActionCable fix)
│
└── PR-8: Inbox Calling Config (provider_config + serializer)
```
---
## Backend PRs
### PR-1: Call Model + Migration + Error Classes ✅ MERGED
> **Status:** Merged via PR #14026 (`feat/voice-call-model` branch).
**Files included:** `db/migrate/20260408170902_create_calls.rb`, `enterprise/app/models/call.rb`, association concerns, `enterprise/lib/whatsapp/call_errors.rb`, `config/features.yml` (whatsapp_call flag definition).
**No action required — already in `develop`.**
---
### PR-2: Meta Cloud API Provider Methods
**Concept:** Low-level HTTP methods that communicate with Meta's WhatsApp Cloud API v22.0 for voice calls. No business logic — just the raw API client layer.
**Files:**
| File | Action | Lines |
|------|--------|-------|
| `enterprise/app/services/whatsapp/providers/whatsapp_cloud_call_methods.rb` | NEW | ~85 |
| `enterprise/app/services/enterprise/whatsapp/providers/whatsapp_cloud_service.rb` | MODIFY | +include |
| `enterprise/app/services/enterprise/whatsapp/facebook_api_client.rb` | MODIFY | +`calls` to webhook fields |
**Methods provided:**
```
pre_accept_call(call_id) → 200 OK (keeps call alive for SDP exchange)
accept_call(call_id, sdp_answer) → 200 OK
reject_call(call_id) → 200 OK
terminate_call(call_id) → 200 OK
initiate_call(to, sdp_offer) → {call_id, sdp_answer, ice_servers}
send_call_permission_request(to) → 200 OK (interactive permission template)
```
**Dependencies:** PR-1 ✅ (merged).
**Risk:** Low. Methods exist but are dormant until consumers ship.
---
### PR-3: Inbound Webhook Pipeline
**Concept:** The complete server-side flow for processing incoming call webhooks from Meta: event routing → call creation → message building → ActionCable broadcasting.
**Files:**
| File | Action | Lines |
|------|--------|-------|
| `enterprise/app/services/whatsapp/incoming_call_service.rb` | NEW | ~211 |
| `enterprise/app/services/whatsapp/call_message_builder.rb` | NEW | ~108 |
| `enterprise/app/services/whatsapp/call_permission_reply_service.rb` | NEW | ~61 |
| `enterprise/app/jobs/enterprise/webhooks/whatsapp_events_job.rb` | NEW | ~40 |
**How it works:**
1. Meta sends webhook with `calls` field → `WhatsappEventsJob` routes to `IncomingCallService`
2. `IncomingCallService` creates/updates `Call` record based on event type (`connect` or `terminate`)
3. `CallMessageBuilder` creates `voice_call` content-type messages in the conversation
4. ActionCable broadcasts `whatsapp_call.incoming` / `whatsapp_call.ended` to the account channel
5. For `call_permission_reply` interactive messages → `CallPermissionReplyService` handles opt-in
**Key detail:** `IncomingCallService` does NOT call Meta's API — it only processes incoming data. No dependency on PR-2.
**Dependencies:** PR-1 ✅ (merged), PR-7 (OSS touchpoints for `prepend_mod_with` hook).
**Risk:** Medium. The webhook event routing (`prepend_mod_with` in the enterprise job) overrides `handle_message_events`. Must ensure `super` delegation doesn't break existing message processing.
---
### PR-4: Call Service + Controller + Routes
**Concept:** The REST API surface for agent-driven call actions (accept, reject, terminate, initiate) and the service orchestrating those actions.
**Files:**
| File | Action | Lines |
|------|--------|-------|
| `enterprise/app/services/whatsapp/call_service.rb` | NEW | ~109 |
| `enterprise/app/controllers/api/v1/accounts/whatsapp_calls_controller.rb` | NEW | ~142 |
| `config/routes.rb` | MODIFY | +resources |
| `enterprise/app/models/enterprise/conversation.rb` | MODIFY | +`allowed_keys?` for `call_status` |
**Endpoints:**
| Method | Path | Action | Description |
|--------|------|--------|-------------|
| GET | `/api/v1/accounts/:account_id/whatsapp_calls/:id` | show | Get call details (SDP offer, ICE servers) |
| POST | `/api/v1/accounts/:account_id/whatsapp_calls/:id/accept` | accept | Accept with SDP answer |
| POST | `/api/v1/accounts/:account_id/whatsapp_calls/:id/reject` | reject | Reject the call |
| POST | `/api/v1/accounts/:account_id/whatsapp_calls/:id/terminate` | terminate | End active call |
| POST | `/api/v1/accounts/:account_id/whatsapp_calls/initiate` | initiate | Start outbound call |
| POST | `/api/v1/accounts/:account_id/whatsapp_calls/:id/upload_recording` | upload_recording | Upload recording blob |
**CallService flow (accept):**
```
pre_accept_call → WebRTC SDP exchange → accept_call (with SDP fix: actpass→active) → update status → broadcast
```
**Dependencies:** PR-1 ✅ (merged), PR-2 (Meta API methods), PR-3 (CallMessageBuilder).
**Risk:** Medium. The controller's `initiate` action handles the permission flow (error 138006 → `send_call_permission_request`). The `upload_recording` action enqueues `CallTranscriptionJob` (PR-5). If PR-5 hasn't merged, temporarily guard the enqueue.
---
### PR-5: Recording Upload + Transcription Pipeline
**Concept:** Async transcription of call recordings using OpenAI Whisper.
**Files:**
| File | Action | Lines |
|------|--------|-------|
| `enterprise/app/services/whatsapp/call_transcription_service.rb` | NEW | ~82 |
| `enterprise/app/jobs/whatsapp/call_transcription_job.rb` | NEW | ~15 |
**Pipeline:**
```
Browser records audio (MediaRecorder) → upload_recording endpoint →
ActiveStorage attachment → CallTranscriptionJob →
CallTranscriptionService → OpenAI Whisper (whisper-1, temp 0.4) →
call.transcript + message.content_attributes updated
```
**Gating:** Requires `captain_integration` feature flag + usage limits check. Inherits from `Llm::LegacyBaseOpenAiService`.
**Error handling:** Retries on `ActiveStorage::FileNotFoundError`, discards on `Faraday::BadRequestError`.
**Dependencies:** PR-1 ✅ (merged). Can merge before or with PR-4.
**Risk:** Low. Fully async, failure-tolerant. No impact on call flow if transcription fails.
---
### PR-6: Feature Flag + Enterprise Gating
**Concept:** Ensure the `whatsapp_call` feature flag is properly checked at all entry points.
**What it covers:**
- Controller: `ensure_whatsapp_call_enabled` before_action
- IncomingCallService: `account.feature_enabled?('whatsapp_call')` check
- CallPermissionReplyService: same check
- Inbox-level toggle: `provider_config['calling_enabled']` on Channel::Whatsapp
> **Note:** The feature flag definition in `config/features.yml` shipped with PR-1. This PR adds the runtime guard logic across services and controllers. May be bundled into PR-4 if too small standalone.
---
### PR-7: OSS Touchpoints
**Concept:** Minimal changes to OSS (non-enterprise) files required for the calling feature to work.
**Files:**
| File | Action | Lines | Change |
|------|--------|-------|--------|
| `app/jobs/webhooks/whatsapp_events_job.rb` | MODIFY | ~10 | Add `prepend_mod_with` hook + extract `handle_message_events` |
| `app/listeners/action_cable_listener.rb` | MODIFY | ~2 | Nil safety fix in `typing_event_listener_tokens` |
**Why separate:** These are the only two OSS Ruby files modified. Keeping them in a dedicated PR makes review clear — reviewers can verify the `super` delegation path and ensure existing message processing isn't broken.
> **Note:** The `handle_message_events` extraction from the webhook job also supports the `smb_message_echoes` feature (PR #13371). If that's already merged to `develop`, this extraction may already exist. Check before creating this PR.
**Dependencies:** None. Should merge early (before PR-3 which depends on the `prepend_mod_with` hook).
**Risk:** Low but critical to verify. The `super` call path must be tested to ensure non-call webhooks still process correctly.
---
### PR-8: Inbox Calling Config (provider_config + serializer)
**Concept:** Backend support for the per-inbox `calling_enabled` toggle in `provider_config` for WhatsApp Cloud channels.
**What it covers:**
- Allow `calling_enabled` in strong params for Channel::Whatsapp updates
- Expose `calling_enabled` in inbox serializer response
- Ensure `provider_config` merge (not overwrite) when updating
**Dependencies:** PR-1 ✅ (merged).
**Risk:** Low. Purely additive config key — no behavior change if not set.
---
## Frontend PR
### PR-9: All Frontend Changes (Combined)
**Concept:** The complete frontend implementation — API client, store, WebRTC engine, UI widgets, ActionCable integration, message bubble enhancements, outbound calling, inbox settings toggle, voice helper guard, i18n, and feature flag registration.
**Files:**
| File | Action | Lines | Layer |
|------|--------|-------|-------|
| `app/javascript/dashboard/api/whatsappCalls.js` | NEW | ~43 | API client |
| `app/javascript/dashboard/stores/whatsappCalls.js` | NEW | ~94 | State management |
| `app/javascript/dashboard/featureFlags.js` | MODIFY | +2 | Feature flag |
| `app/javascript/dashboard/i18n/locale/en/whatsappCall.json` | NEW | ~20 | i18n |
| `app/javascript/dashboard/i18n/locale/en/index.js` | MODIFY | +1 | i18n |
| `app/javascript/dashboard/helper/voice.js` | MODIFY | ~20 | Voice guard |
| `app/javascript/dashboard/composables/useWhatsappCallSession.js` | NEW | ~406 | WebRTC engine |
| `app/javascript/dashboard/helper/actionCable.js` | MODIFY | +5 handlers | Real-time events |
| `app/javascript/dashboard/components/widgets/WhatsappCallWidget.vue` | NEW | ~216 | Call widget UI |
| `app/javascript/dashboard/routes/dashboard/Dashboard.vue` | MODIFY | +import | Widget mount |
| `app/javascript/dashboard/components-next/message/bubbles/VoiceCall.vue` | MODIFY | ~145 | Message bubble |
| `app/javascript/dashboard/i18n/locale/en/conversation.json` | MODIFY | +6 keys | i18n |
| `app/javascript/dashboard/components/widgets/conversation/ConversationHeader.vue` | MODIFY | ~130 | Outbound calling |
| `app/javascript/dashboard/routes/dashboard/settings/inbox/settingsPage/ConfigurationPage.vue` | MODIFY | ~20 | Inbox settings |
| `app/javascript/dashboard/i18n/locale/en/inboxMgmt.json` | MODIFY | +3 keys | i18n |
**Key components:**
1. **API Client** — `show`, `accept`, `reject`, `terminate`, `initiate`, `uploadRecording`
2. **Pinia Store** — Module-scoped WebRTC objects (non-serializable) outside reactive state
3. **WebRTC Composable** — `useWhatsappCallSession()`, `acceptWhatsappCallById()`, `startCallRecording()`
4. **ActionCable Events** — `incoming`, `accepted`, `ended`, `outbound_connected`, `permission_granted`
5. **Floating Widget** — Incoming/active call states with accept/reject/mute/hangup
6. **VoiceCall Bubble** — Accept button, recording player, transcript toggle, "answered by" display
7. **Outbound UI** — Phone icon in ConversationHeader, full WebRTC offer/answer flow
8. **Inbox Toggle** — `calling_enabled` checkbox in WhatsApp Cloud inbox settings
9. **Voice Guard** — Prevents WhatsApp call messages from triggering Twilio store
**Backend prerequisites:** PR-4 (REST endpoints), PR-3 (ActionCable broadcasts), PR-8 (inbox config).
**Risk:** ⚠️ HIGH — Largest PR. Review focus areas:
- Duplicated ICE gathering logic in ConversationHeader vs composable
- Missing `onUnmounted` cleanup for outbound calls in ConversationHeader
- `console.log` left in ConversationHeader ICE state logging
- Hardcoded STUN server with no backend-provided config
- Recording mimeType `audio/webm;codecs=opus` with no browser feature detection
- Bare string in ActionCable `onWhatsappCallPermissionGranted` (should use i18n)
- Frontend feature flag declared but unused (gated server-side only)
**Estimated size:** ~1,200 lines across 15 files.
---
## Recommended Merge Sequence
### Phase 1: Foundation (Week 1)
| Order | PR | Type | Depends On | Est. Size |
|-------|-----|------|------------|-----------|
| — | **PR-1:** Call Model + Migration | Backend | None | ✅ MERGED |
| 1 | **PR-7:** OSS Touchpoints | Backend | None | ~12 lines |
| 2 | **PR-8:** Inbox Calling Config | Backend | PR-1 ✅ | ~30 lines |
### Phase 2: Backend Services (Week 1-2)
| Order | PR | Type | Depends On | Est. Size |
|-------|-----|------|------------|-----------|
| 3 | **PR-2:** Meta API Provider Methods | Backend | PR-1 ✅ | ~90 lines |
| 4 | **PR-3:** Inbound Webhook Pipeline | Backend | PR-1 ✅, PR-7 | ~420 lines |
| 5 | **PR-5:** Transcription Pipeline | Backend | PR-1 ✅ | ~97 lines |
| 6 | **PR-6:** Feature Flag Gating | Backend | PR-1 ✅ | ~20 lines |
### Phase 3: Backend API + Frontend (Week 2-3)
| Order | PR | Type | Depends On | Est. Size |
|-------|-----|------|------------|-----------|
| 7 | **PR-4:** Call Service + Controller + Routes | Backend | PR-2, PR-3, PR-5 | ~260 lines |
| 8 | **PR-9:** All Frontend Changes | Frontend | PR-4, PR-3, PR-8 | ~1,200 lines |
---
## Parallel Merge Opportunities
These PRs have no dependencies on each other and can be reviewed/merged in parallel:
- **PR-7, PR-8** — independent foundation PRs
- **PR-2, PR-3, PR-5, PR-6** — all depend only on PR-1 (merged) and/or PR-7
---
## Issues Found During Analysis
| # | Issue | Severity | PR Affected |
|---|-------|----------|-------------|
| 1 | **Duplicated ICE gathering logic** — `waitForOutboundIceGathering` in ConversationHeader vs `waitForIceGatheringComplete` in composable | Medium | PR-9 |
| 2 | **No `onUnmounted` cleanup** for outbound calls in ConversationHeader | Medium | PR-9 |
| 3 | **Feature flag unused on frontend** — `FEATURE_FLAGS.WHATSAPP_CALL` declared but no component checks it | Low | PR-9 |
| 4 | **Bare string in ActionCable** — `onWhatsappCallPermissionGranted` uses template literal instead of i18n key | Low | PR-9 |
| 5 | **Hardcoded STUN server** — `stun:stun.l.google.com:19302` with no backend-provided config | Low | PR-9 |
| 6 | **Recording mimeType assumption** — `audio/webm;codecs=opus` with no browser feature detection | Low | PR-9 |
| 7 | **`console.log` left in** — ICE state logging in ConversationHeader | Low | PR-9 |
---
## Cherry-Pick Strategy
Since the branch has ~270+ commits (including develop merges), cherry-picking individual commits won't work cleanly. Instead:
1. **For each PR**, create a new branch from `develop`
2. **Copy the specific files** from `feat/whatsapp-call` using:
```bash
git checkout feat/whatsapp-call -- path/to/file1 path/to/file2
```
3. **For modified files** (routes.rb, actionCable.js, etc.), manually apply only the relevant diff hunks
4. **Run tests** for each PR independently before opening
---
## Testing Checklist Per PR
| PR | Test Strategy |
|----|---------------|
| PR-2 | Unit test Meta API methods with WebMock/VCR stubs |
| PR-3 | Integration test: simulate Meta webhook payload → verify Call record + message created |
| PR-4 | Request specs: hit each endpoint, verify response + side effects |
| PR-5 | Unit test: mock OpenAI API, verify transcript stored on call + message |
| PR-6 | Verify feature flag gating blocks unauthorized accounts |
| PR-7 | Verify existing WhatsApp message webhook processing still works |
| PR-8 | Verify inbox serializer includes `calling_enabled` |
| PR-9 | `pnpm test` on store + manual E2E: inbound call, outbound call, recording, settings toggle |
docs/WHATSAPP_CALL_SERVER_RELAY_ARCHITECTURE.md
-694
View File
@@ -1,694 +0,0 @@
# Frontend Architecture: Server-Side WebRTC WhatsApp Calling
> Design document for migrating WhatsApp calling from browser-side WebRTC to server-side WebRTC with browser audio relay.
---
## 1. Architecture Overview
### Current model (being replaced)
```
Agent Browser Meta Media Servers
┌─────────────────────┐ ┌──────────────┐
│ RTCPeerConnection │◄═══════►│ SRTP Audio │
│ SDP offer/answer │ direct │ │
│ ICE candidates │ media │ │
│ MediaRecorder │ │ │
│ AudioContext mixer │ │ │
└─────────────────────┘ └──────────────┘
▲
│ ActionCable (signaling only)
▼
┌─────────────────────┐
│ Chatwoot Backend │
│ (relay SDP/ICE) │
└─────────────────────┘
```
### New model
```
Agent Browser Chatwoot Backend Meta Media Servers
┌──────────────────┐ WS ┌─────────────────┐ SRTP ┌──────────────┐
│ getUserMedia() │────────────►│ Audio Ingest │ │ │
│ (mic capture) │ agent mic │ │ │ │
│ │ │ RTCPeerConn │◄════════►│ Media │
│ AudioContext │◄────────────│ SDP/ICE │ direct │ Servers │
│ (playback) │ remote │ MediaRecorder │ media │ │
│ │ audio │ (server-side) │ │ │
│ UI Controls │ │ │ │ │
│ (mute/hangup) │◄───────────►│ ActionCable │ │ │
│ │ signaling │ (call events) │ │ │
└──────────────────┘ └─────────────────┘ └──────────────┘
```
Key changes:
- **WebRTC lives on the server.** The backend holds the RTCPeerConnection to Meta.
- **Browser sends mic audio** to the server over a dedicated binary WebSocket.
- **Server relays remote audio** back to the browser over the same WebSocket.
- **Recording is server-side.** No client-side MediaRecorder, no upload step.
- **Page reload does not kill the call.** The server-side connection persists; the browser just reconnects its audio stream.
---
## 2. Recommended Audio Relay Mechanism
### Evaluation of options
| Mechanism | Latency | Complexity | Browser Support | Reconnection | Verdict |
|-----------|---------|-----------|----------------|--------------|---------|
| **Dedicated binary WebSocket** | 20-50ms | Low | Universal | Trivial (new WS) | **Recommended** |
| Secondary WebRTC (browser to server) | 5-20ms | High | Universal | Hard (new SDP) | Over-engineered |
| WebTransport | 5-15ms | Medium | Chrome/Edge only | Medium | Not ready |
| ActionCable binary frames | 30-80ms | Low | Universal | Free (existing) | Too slow, not designed for media |
### Recommendation: Dedicated binary WebSocket
**Why not reuse ActionCable?** ActionCable is JSON-framed, multiplexed, and adds overhead for binary payloads. Audio needs a dedicated low-latency binary channel with no framing overhead.
**Why not a second WebRTC connection?** The entire point of this migration is to remove WebRTC complexity from the browser. Adding a browser-to-server WebRTC leg reintroduces SDP negotiation, ICE gathering, and DTLS setup -- the exact things we are eliminating.
**Why not WebTransport?** Firefox and Safari do not yet have stable support. The latency improvement over WebSocket (5-15ms vs 20-50ms) does not justify excluding a third of agents.
### Audio relay protocol design
```
Dedicated WebSocket: wss://{host}/cable/audio?call_id={id}&token={jwt}
Frame format (binary):
┌──────────┬──────────┬─────────────────────────┐
│ type (1B)│ seq (2B) │ payload (variable) │
├──────────┼──────────┼─────────────────────────┤
│ 0x01 │ uint16 │ Opus frame (agent mic) │ browser → server
│ 0x02 │ uint16 │ Opus frame (remote) │ server → browser
│ 0x03 │ - │ JSON control message │ bidirectional
└──────────┴──────────┴─────────────────────────┘
Audio encoding:
- Codec: Opus (native to WebRTC, supported by AudioEncoder API)
- Sample rate: 48kHz mono (Opus default)
- Frame duration: 20ms (960 samples per frame)
- Bitrate: 24-32 kbps (speech-optimized)
- Frames per WebSocket message: 1 (20ms per message = 50 messages/sec)
Control messages (type 0x03):
- { "action": "mute" }
- { "action": "unmute" }
- { "action": "heartbeat" }
- { "action": "reconnected", "resumeFrom": seq }
```
**Why Opus?** It is the codec Meta uses for WhatsApp call audio. The server receives Opus from Meta's SRTP stream and can forward it directly to the browser without transcoding. The browser can also encode mic input as Opus using the WebCodecs `AudioEncoder` API (Chrome 94+, Firefox 130+, Safari 16.4+).
**Fallback for older browsers:** If `AudioEncoder` is unavailable, fall back to sending raw PCM Int16 at 16kHz (32 KB/s). The server transcodes to Opus before injecting into the WebRTC session.
---
## 3. Component Tree
```
App.vue
└── DashboardLayout.vue
├── ConversationHeader.vue
│ └── [phone icon button] ──► calls store.initiateCall()
│
├── MessageBubble (VoiceCall.vue)
│ └── [accept/join button] ──► calls store.acceptCall()
│
└── CallWidget.vue (fixed position, bottom-right)
├── IncomingCallCard.vue (for each ringing call)
│ ├── Avatar + caller info
│ ├── Accept button ──► calls store.acceptCall()
│ └── Reject button ──► calls store.rejectCall()
│
├── ActiveCallCard.vue (single active call)
│ ├── Avatar + caller info
│ ├── Duration timer (formattedDuration from store)
│ ├── Mute toggle ──► calls store.toggleMute()
│ ├── Hangup button ──► calls store.endCall()
│ └── ReconnectingBanner.vue (shown during audio reconnection)
│
└── CallErrorBanner.vue
```
### What changed vs current
- **Removed:** No WebRTC logic in any component. Components are now pure UI.
- **Simplified:** ConversationHeader no longer creates RTCPeerConnection or manages SDP. It calls a single store action.
- **Added:** `ReconnectingBanner` sub-component for the reconnection state.
- **Same:** WhatsappCallWidget stays as the floating overlay. VoiceCall.vue bubble stays as the message thread display.
---
## 4. Composable Design
The current monolithic `useWhatsappCallSession.js` (406 lines) is split into three focused composables:
### 4a. `useCallAudioStream.js` -- audio capture and playback
This composable owns the dedicated WebSocket, mic capture, and audio playback. It has zero knowledge of call signaling or UI state.
```
Module-level state (singleton, survives component remounts):
- audioSocket: WebSocket | null
- audioContext: AudioContext | null
- micStream: MediaStream | null
- audioEncoder: AudioEncoder | null (WebCodecs)
- playbackNode: AudioWorkletNode | null
- sequenceNumber: number
- isConnected: ref(false)
- isCapturing: ref(false)
Exported function: useCallAudioStream()
Returns:
// State
isAudioConnected: Ref<boolean>
isCapturing: Ref<boolean>
isReconnecting: Ref<boolean>
audioLevel: Ref<number> // 0-1, for visual feedback
// Actions
connect(callId: string): Promise<void>
1. Open WS to /cable/audio?call_id={callId}&token={jwt}
2. Create AudioContext (48kHz)
3. getUserMedia({ audio: true })
4. Pipe mic → AudioWorkletNode (capture processor) → AudioEncoder → WS send
5. Register WS.onmessage handler: decode Opus frames → playback buffer
6. Start playback via AudioWorkletNode (playback processor)
disconnect(): void
1. Close WS
2. Stop mic tracks
3. Close AudioContext
4. Reset all state
setMuted(muted: boolean): void
1. If muted: stop sending mic frames (but keep mic open for fast unmute)
2. If unmuted: resume sending
// No server round-trip needed. Simply stop emitting frames.
reconnect(callId: string): Promise<void>
1. Disconnect existing
2. Connect fresh (server already has the call)
3. Resume audio from current server buffer position
```
**AudioWorklet processors** (two small files):
`mic-capture-processor.js` -- runs on the audio thread:
```js
// Receives Float32 PCM from getUserMedia
// Posts ArrayBuffer to main thread for encoding
class MicCaptureProcessor extends AudioWorkletProcessor {
process(inputs) {
const input = inputs[0][0]; // mono channel
if (input) {
this.port.postMessage(input.buffer, [input.buffer]);
}
return true;
}
}
registerProcessor('mic-capture', MicCaptureProcessor);
```
`audio-playback-processor.js` -- runs on the audio thread:
```js
// Receives decoded PCM buffers via port.postMessage
// Writes them into output for playback
// Maintains a small jitter buffer (60-100ms)
class AudioPlaybackProcessor extends AudioWorkletProcessor {
constructor() {
super();
this.buffer = []; // ring buffer of Float32Arrays
this.port.onmessage = (e) => this.buffer.push(e.data);
}
process(outputs) {
const output = outputs[0][0];
if (this.buffer.length > 0) {
const frame = this.buffer.shift();
output.set(frame.subarray(0, output.length));
}
return true;
}
}
registerProcessor('audio-playback', AudioPlaybackProcessor);
```
### 4b. `useCallSession.js` -- call lifecycle and UI state
This replaces the current `useWhatsappCallSession.js`. It orchestrates call signaling (via REST API) and delegates audio to `useCallAudioStream`. No WebRTC code.
```
Exported function: useCallSession()
Internally uses:
- useWhatsappCallsStore() (Pinia)
- useCallAudioStream()
- Timer helper
Returns:
// Read-only state (delegated from store)
activeCall: ComputedRef<CallData | null>
incomingCalls: ComputedRef<CallData[]>
hasActiveCall: ComputedRef<boolean>
hasIncomingCall: ComputedRef<boolean>
firstIncomingCall: ComputedRef<CallData | null>
isOutboundRinging: ComputedRef<boolean>
// Session state
isAccepting: Ref<boolean>
isMuted: Ref<boolean>
isReconnecting: Ref<boolean> // NEW: true during page-reload rejoin
callError: Ref<string | null>
formattedCallDuration: ComputedRef<string>
// Actions
acceptCall(call): Promise<void>
1. POST /whatsapp_calls/:id/accept (NO SDP -- server handles WebRTC)
2. store.setActiveCall(call)
3. audioStream.connect(call.id)
4. timer.start()
rejectCall(call): Promise<void>
1. POST /whatsapp_calls/:id/reject
2. store.removeIncomingCall(call.callId)
endCall(): Promise<void>
1. audioStream.disconnect()
2. POST /whatsapp_calls/:id/terminate
3. store.clearActiveCall()
4. timer.stop()
toggleMute(): void
1. audioStream.setMuted(!isMuted.value)
2. isMuted.value = !isMuted.value
initiateCall(conversationId): Promise<void> // NEW: moved from ConversationHeader
1. POST /whatsapp_calls/initiate (NO SDP offer -- server creates its own)
2. store.setActiveCall({ status: 'ringing', direction: 'outbound', ... })
3. Wait for ActionCable 'whatsapp_call.outbound_connected' event
4. audioStream.connect(call.id)
5. timer.start()
rejoinCall(callId): Promise<void> // NEW: reconnection after page reload
1. GET /whatsapp_calls/:id → verify status is 'accepted' or 'in_progress'
2. store.setActiveCall(callData)
3. audioStream.connect(callId)
4. Fetch elapsed duration from server to resume timer
dismissIncomingCall(call): void
1. store.removeIncomingCall(call.callId)
```
### 4c. `useCallReconnection.js` -- handles page reload during active call
```
Exported function: useCallReconnection()
Logic (runs on mount):
1. On app startup, check: GET /whatsapp_calls/active
- New API endpoint that returns the agent's current active call (if any)
2. If an active call exists:
- Set isReconnecting = true
- Call callSession.rejoinCall(callId)
- Set isReconnecting = false
3. Register visibility change listener:
- On document becoming visible after being hidden, verify audio WS is healthy
- If WS is closed, trigger reconnect
```
---
## 5. Pinia Store Schema
The store becomes drastically simpler. All non-serializable objects (RTCPeerConnection, MediaStream, AudioContext) move out of the store entirely -- they live in `useCallAudioStream` at module scope.
### New store: `stores/whatsappCalls.js`
```js
// NO module-scoped WebRTC objects. No outboundCall state.
// The store is now purely serializable call metadata.
defineStore('whatsappCalls', {
state: () => ({
incomingCalls: [], // CallData[]
activeCall: null, // CallData | null
callTimerOffset: 0, // seconds already elapsed (for reconnection)
}),
getters: {
hasIncomingCall: (state) => state.incomingCalls.length > 0,
hasActiveCall: (state) => state.activeCall !== null,
hasWhatsappCall: (state) => state.incomingCalls.length > 0 || state.activeCall !== null,
firstIncomingCall: (state) => state.incomingCalls[0] || null,
},
actions: {
addIncomingCall(callData) { ... },
removeIncomingCall(callId) { ... },
setActiveCall(callData) { ... },
clearActiveCall() { ... },
markActiveCallConnected() { ... },
handleCallAcceptedByOther(callId) { ... },
handleCallEnded(callId) { ... },
setTimerOffset(seconds) { this.callTimerOffset = seconds; },
},
});
```
### CallData type definition
```ts
interface CallData {
id: number; // server-side Call record ID
callId: string; // Meta's call_id
direction: 'inbound' | 'outbound';
status: 'ringing' | 'connected' | 'reconnecting';
inboxId: number;
conversationId: number;
caller: {
name?: string;
phone?: string;
avatar?: string;
};
// REMOVED: sdpOffer, iceServers -- no longer sent to browser
}
```
### What was removed from the store
| Old | New | Why |
|-----|-----|-----|
| `outboundCall` (module-scoped `{ pc, stream, audio, callId }`) | Gone | No WebRTC objects in browser |
| `getOutboundCallState()` | Gone | No peer connection to query |
| `setOutboundCallProperty()` | Gone | No peer connection to mutate |
| `cleanupOutboundCall()` | Gone | Audio cleanup is in `useCallAudioStream.disconnect()` |
| `cleanupCallback` | Gone | No need for cross-concern cleanup registration |
| `registerCleanupCallback()` | Gone | Store actions directly call composable |
### What was added
| New | Why |
|-----|-----|
| `callTimerOffset` | When reconnecting after page reload, the server tells us how many seconds the call has been active. The timer starts from this offset instead of 0. |
---
## 6. Reconnection Flow
This is the primary benefit of the server-side architecture. The server's WebRTC connection to Meta persists across browser page loads.
### Scenario: Agent refreshes page during active call
```
Timeline:
─────────────────────────────────────────────────────────────────►
1. Agent is on an active call (audio flowing)
2. Agent hits F5 (page reload)
├── beforeunload fires
│ └── Audio WS closes (but we do NOT terminate the call)
│ Unlike current code which calls terminateCallOnUnload()
├── Server detects WS disconnect
│ └── Server continues holding the Meta WebRTC session
│ └── Server buffers incoming remote audio (or drops it, brief gap is OK)
└── Browser unloads
3. Page reloads, Vue app mounts
├── useCallReconnection() runs on mount
│ └── GET /whatsapp_calls/active
│ Response: { id: 42, call_id: "abc", status: "accepted",
│ direction: "inbound", elapsed_seconds: 47, ... }
│
├── Store updated: setActiveCall(callData)
│ UI immediately shows CallWidget with "Reconnecting..." banner
│
├── callSession.rejoinCall(42)
│ ├── audioStream.connect(42)
│ │ ├── New WS to /cable/audio?call_id=42
│ │ ├── getUserMedia() -- browser may re-prompt for mic
│ │ ├── AudioContext + worklets initialized
│ │ └── Audio flowing again
│ │
│ └── Timer resumes from elapsed_seconds (47)
│ formattedCallDuration shows "00:47" and counting
│
└── ReconnectingBanner disappears, ActiveCallCard shows normally
4. Call continues as if nothing happened
Total audio gap: ~2-4 seconds (page load time)
```
### Scenario: Network blip (WebSocket drops briefly)
```
1. Audio WS disconnects unexpectedly
2. useCallAudioStream detects WS close
├── isReconnecting = true
├── UI shows "Reconnecting..." in ActiveCallCard
└── Start reconnection with exponential backoff:
attempt 1: wait 500ms → try WS connect
attempt 2: wait 1000ms → try WS connect
attempt 3: wait 2000ms → try WS connect
max 5 attempts, then show error
3. WS reconnects
├── isReconnecting = false
├── Audio resumes
└── UI returns to normal
4. If all 5 attempts fail:
├── Show error: "Audio connection lost. Call is still active on server."
└── Offer "Reconnect" button that retries
```
### Scenario: Agent opens a second tab
```
1. Call is active in Tab A
2. Agent opens Tab B (or navigates to Chatwoot in new tab)
├── useCallReconnection() runs
│ └── GET /whatsapp_calls/active → returns active call
│
├── Two options (backend decides):
│ a) TRANSFER audio to new tab: Server closes Tab A's audio WS,
│ Tab B becomes the audio source. Tab A shows "Call moved to another tab."
│ b) BLOCK: Return { active: true, owned_by_other_session: true }
│ Tab B shows the call widget in "view only" mode (timer, no controls)
│
└── Recommendation: Option (a) -- transfer. Matches user intent
(they probably want to continue in the new tab).
```
---
## 7. File Structure
### Files to create (new)
```
app/javascript/dashboard/
├── composables/
│ ├── useCallAudioStream.js # Audio capture, WS relay, playback
│ ├── useCallSession.js # Call lifecycle (accept/reject/end/initiate)
│ └── useCallReconnection.js # Page-reload reconnection logic
│
├── workers/
│ ├── mic-capture-processor.js # AudioWorklet: mic → main thread
│ └── audio-playback-processor.js # AudioWorklet: main thread → speaker
│
└── helpers/
└── callAudioCodec.js # WebCodecs Opus encode/decode helpers
```
### Files to modify
```
app/javascript/dashboard/
├── composables/
│ └── useWhatsappCallSession.js # DELETE entirely (replaced by useCallSession.js)
│
├── stores/
│ └── whatsappCalls.js # Simplify: remove all WebRTC state/helpers
│
├── api/
│ └── whatsappCalls.js # Modify: remove SDP params, add active() endpoint
│
├── helper/
│ └── actionCable.js # Modify: simplify event handlers (no SDP relay)
│
├── components/widgets/
│ ├── WhatsappCallWidget.vue # Minor: use new composable, add reconnection UI
│ └── conversation/
│ └── ConversationHeader.vue # Major simplification: remove all WebRTC code
│
└── components-next/message/bubbles/
└── VoiceCall.vue # Minor: acceptWhatsappCallById → store action
```
### Files to delete
```
(none explicitly deleted as files, but the following become unnecessary
and their content is removed or replaced:)
- All RTCPeerConnection code in useWhatsappCallSession.js → replaced
- All SDP/ICE code in ConversationHeader.vue → replaced
- startCallRecording / stopAndUploadRecording → gone (server-side recording)
- terminateCallOnUnload → replaced with "do nothing" (call persists)
```
---
## 8. API Changes Required
The REST API needs small changes to support the new model:
### Modified endpoints
| Endpoint | Old | New |
|----------|-----|-----|
| `POST /:id/accept` | Sends `{ sdp_answer }` | Sends `{}` (no SDP, server handles it) |
| `POST /initiate` | Sends `{ conversation_id, sdp_offer }` | Sends `{ conversation_id }` (no SDP) |
| `POST /:id/upload_recording` | Browser uploads webm blob | **Removed** -- server records directly |
### New endpoints
| Endpoint | Purpose |
|----------|---------|
| `GET /whatsapp_calls/active` | Returns the current agent's active call (if any). Used for reconnection on page load. Returns `null` if no active call. |
| `WS /cable/audio?call_id={id}&token={jwt}` | Dedicated binary WebSocket for audio relay. Separate from ActionCable. |
### Modified ActionCable events
| Event | Old payload | New payload |
|-------|-------------|-------------|
| `whatsapp_call.incoming` | `{ ..., sdp_offer, ice_servers }` | `{ ..., }` (no SDP/ICE -- browser does not need them) |
| `whatsapp_call.outbound_connected` | `{ call_id, sdp_answer }` | `{ call_id }` (no SDP -- just signals "ready for audio WS") |
---
## 9. ActionCable Handler Changes
The `actionCable.js` handlers simplify significantly:
```
Current handlers (what changes):
onWhatsappCallIncoming:
OLD: Store sdpOffer + iceServers in incoming call data
NEW: Store only metadata (id, callId, direction, caller, conversationId)
No sdpOffer, no iceServers
onWhatsappCallAccepted:
OLD: Same
NEW: Same (no change -- purely a signaling event)
onWhatsappCallEnded:
OLD: Calls store.handleCallEnded which triggers cleanupCallback (WebRTC teardown)
NEW: Calls store.handleCallEnded which is now just state cleanup
Audio cleanup happens separately via useCallAudioStream detecting WS close
onWhatsappCallOutboundConnected:
OLD: Gets outbound PC, calls pc.setRemoteDescription(sdp_answer)
NEW: Calls store.markActiveCallConnected()
Triggers useCallSession to open the audio WS
No SDP handling at all
onWhatsappCallPermissionGranted:
OLD: Same
NEW: Same (no change -- purely a UI notification)
```
---
## 10. Comparison: Old vs New
### Code complexity
| Aspect | Old (browser WebRTC) | New (server relay) |
|--------|---------------------|-------------------|
| **useWhatsappCallSession.js** | 406 lines, WebRTC + recording + SDP + ICE | ~80 lines, pure call lifecycle orchestration |
| **ConversationHeader.vue (call code)** | ~100 lines of WebRTC setup in component | ~10 lines: single `store.initiateCall()` call |
| **Pinia store** | 94 lines + module-scoped `outboundCall` with PC/Stream | ~60 lines, purely serializable state |
| **actionCable.js (call handlers)** | 50 lines with SDP relay logic | ~25 lines, metadata-only event handling |
| **AudioWorklet processors** | N/A | ~30 lines each (two files), standard pattern |
| **useCallAudioStream.js** | N/A (was inline WebRTC) | ~150 lines, focused audio I/O |
| **Total frontend call code** | ~650 lines, scattered across 5 files | ~400 lines, organized in 3 composables + 2 worklets |
### Capability comparison
| Capability | Old | New |
|-----------|-----|-----|
| Call survives page reload | No | Yes |
| Call survives network blip | No (ICE restart needed) | Yes (WS reconnect) |
| Recording reliability | Low (browser crash = lost) | High (server-side) |
| Multi-tab support | Not possible | Transfer or view-only |
| Agent on mobile browser | Fragile (background tab kills WebRTC) | Better (WS more resilient) |
| TURN server needed | Yes (for restrictive NAT) | No (server has public IP) |
| Browser API surface | RTCPeerConnection, MediaRecorder, AudioContext, ICE, SDP, SRTP | getUserMedia, AudioContext, WebSocket, WebCodecs |
| Codec flexibility | Locked to WebRTC negotiation | Server can transcode |
### Latency comparison
| Path | Old | New |
|------|-----|-----|
| Agent mic to customer | ~50ms (direct P2P via SRTP) | ~70-100ms (+20-50ms WS hop to server) |
| Customer to agent speaker | ~50ms (direct P2P via SRTP) | ~70-100ms (+20-50ms WS hop from server) |
The added ~20-50ms per direction is imperceptible in voice calls (human perception threshold for audio delay is ~150ms).
### Failure mode comparison
| Failure | Old | New |
|---------|-----|-----|
| Browser crashes | Call dead, recording lost | Call continues on server, agent can rejoin |
| Network drops 5s | ICE restart attempt (often fails) | WS reconnect, brief audio gap |
| Agent closes tab | Call terminated (beforeunload) | Call persists, agent can rejoin (or timeout) |
| Server crashes | Call continues (P2P) | Call dead (single point of failure) |
The server becoming a single point of failure is mitigated by the fact that it already is the single point of failure for signaling. If the server goes down in the old model, the agent cannot accept new calls or initiate calls anyway. The incremental risk is that an *in-progress* call dies, which is manageable with standard server redundancy.
---
## 11. Migration Strategy
### Phase 1: Backend (prerequisite)
- Server-side WebRTC: RTCPeerConnection to Meta from the backend process
- Audio WebSocket endpoint: `/cable/audio` with binary frame handling
- Server-side recording with the same Opus stream
- New `GET /whatsapp_calls/active` endpoint
- Modified `POST /accept` and `POST /initiate` (no SDP params)
### Phase 2: Frontend (this document)
- Create `useCallAudioStream.js` + AudioWorklet processors
- Create `useCallSession.js` to replace `useWhatsappCallSession.js`
- Create `useCallReconnection.js`
- Simplify Pinia store
- Simplify ActionCable handlers
- Update CallWidget with reconnection UI
- Remove all WebRTC code from ConversationHeader
### Phase 3: Cleanup
- Remove `startCallRecording` / `stopAndUploadRecording` exports
- Remove `uploadRecording` API method
- Remove `sdpOffer` / `iceServers` from all frontend types
- Feature flag: `whatsapp_call_server_relay` to toggle between old/new during rollout
---
## 12. Open Questions for Backend Team
1. **Audio WebSocket authentication**: Should we use a short-lived JWT in the WS URL query param, or perform auth in the first WS frame? JWT in URL is simpler but appears in server logs.
2. **Server-side timeout**: When the browser disconnects the audio WS (page reload), how long should the server hold the Meta WebRTC session before auto-terminating? Recommendation: 30 seconds.
3. **Audio buffering during reconnection**: Should the server buffer remote audio during a WS disconnect and replay it when the browser reconnects? Or just drop those frames? Recommendation: Drop -- a 2-4 second gap is acceptable, buffering adds complexity.
4. **Opus passthrough vs transcode**: Can the server forward Meta's Opus frames directly to the browser WS, or does the SRTP decryption produce raw PCM that needs re-encoding? Direct passthrough is ideal (zero CPU cost).
5. **Multi-process deployment**: If Chatwoot runs multiple Rails processes/pods, how does the audio WS route to the process that holds the RTCPeerConnection? Sticky sessions? Redis pub/sub relay? This is a backend architecture question but affects the WS endpoint design.
docs/whatsapp-calling.md
-1180
View File
File diff suppressed because it is too large Load Diff