This improves the Captain overview by loading reporting metrics and FAQ
stats from separate endpoints. Range changes now refresh only the
metrics, while reopen-rate calculation reuses the resolved conversation
count to avoid redundant database queries.
## What changed
- Split Captain overview metrics and FAQ stats into separate APIs.
- Fetch FAQ stats independently from range-based metrics.
- Reuse resolved conversation totals when calculating reopen rate.
- Skip the reopen query when there are no resolved conversations.
Records a `Captain::Session` row for every Captain V2 assistant response
delivered in a conversation, so we can show how a response was generated
and report on credit, FAQ, and document usage. Stacked on #14970 (the
`captain_sessions` model).
## What changed
- `FaqLookupTool` now records the retrieved FAQ ids (and their backing
document ids) into the shared run state, accumulated across tool calls.
- `AgentRunnerService` exposes the raw ai-agents run result via
`last_run_result`; the `generate_response` return shape is unchanged, so
the playground path is unaffected.
- New `Captain::Assistant::SessionCaptureService` builds the session:
scenario resolved from the answering agent name, model from
`assistant.agent_model`, token usage plus the trimmed current-turn
conversation history stored in `run_context`.
- `ResponseBuilderJob` captures after delivery: `credits_consumed`
mirrors the actual charge (1.0 for a billed response, 0.0 for handoffs,
where the session points at the customer-facing handoff message).
Capture runs outside the delivery transaction and swallows its own
failures, so a logging bug can never block or roll back a customer
reply.
V1 responses and copilot are out of scope; copilot capture comes next.
## How to test
On an account with `captain_integration_v2` enabled and an inbox
connected to an assistant with approved FAQs, send a customer message on
a pending conversation. After the assistant replies, a
`Captain::Session` row should exist with the conversation as subject,
the reply message as result, the FAQs/documents used, and the run
context for that turn. Asking for a human agent should produce a
zero-credit session pointing at the handoff message.
<img width="2428" height="1058" alt="CleanShot 2026-07-15 at 17 25
40@2x"
src="https://github.com/user-attachments/assets/d8e44923-c17b-494f-8c33-c8fa4219438c"
/>
Conversation attachment uploads now go through the same authentication
that every other account-scoped API endpoint uses. Agents continue to
attach files exactly as before, and the upload request is now tied to
the agent's dashboard session instead of a separately serialized access
token.
Because the upload request is now authenticated, the dashboard proves
the agent's session directly instead of passing
`currentUser.access_token`. This keeps uploads working alongside the
profile access-token changes in #14973, including on accounts where that
token is serialized as empty.
## What changed
- `Api::V1::Accounts::Conversations::DirectUploadsController` now runs
the standard account auth stack: API access token when the
`api_access_token` header is present, dashboard session
(devise-token-auth) otherwise, with agent-bot tokens rejected.
Previously it inherited `ActiveStorage::DirectUploadsController`
directly and did not run any authentication.
- `EnsureCurrentAccountHelper#ensure_current_account` now returns `401`
when a request has neither an authenticated user nor a bot resource,
instead of continuing. This closes the same gap for any controller that
relies on the helper.
- The dashboard direct-upload paths (`useFileUpload.js` and the legacy
`fileUploadMixin.js`) now attach the agent's session headers to the
upload request via a new `directUploadsHelper.js`, instead of sending
`currentUser.access_token`.
## How to test
1. As a logged-in agent, open a conversation and attach a file. Upload
should succeed as before, on installs with direct uploads enabled.
2. Confirm attachments still work for an agent on an account whose
profile access token is not serialized (e.g. a Cloud plan without
`api_and_webhooks`).
3. Send a `POST` to
`/api/v1/accounts/:account_id/conversations/:conversation_id/direct_uploads`
with no credentials, an empty `api_access_token`, or an invalid token,
and confirm it returns `401`.
4. Confirm a valid agent of the account (via API token or session) gets
`200`, while an agent of a different account gets `401`.
This gates API-token access and outgoing account webhooks behind the
`api_and_webhooks` account feature introduced in #14972. On Chatwoot
Cloud, Hacker accounts lose token-authenticated account API access and
account webhook delivery, while paid accounts retain them through the
billing-plan feature reconcile. Community and self-hosted installations
continue to work without any upgrade-time interruption.
## What changed
- Added `Account#api_and_webhooks_enabled?` as the single backend kill
switch. Core returns enabled; the Enterprise override consults the
account flag on Chatwoot Cloud and remains enabled off-Cloud.
- Account-scoped v1 and v2 requests authenticated with a user or
agent-bot API token now return `403 Forbidden` when the feature is
disabled. Invalid tokens still return 401, and dashboard session
requests are unaffected.
- Profile responses return an empty access token when none of the user's
accounts has access. The stored token is preserved, and the profile UI
disables its token controls with paid-plan copy on Cloud.
- Account webhook delivery stops when the feature is disabled. Webhook
CRUD remains available to session-authenticated dashboard requests,
API-inbox webhooks continue to be delivered, and the Cloud dashboard
shows a webhook paywall instead of the webhook list.
- Removed the database backfill migration. Existing paid Cloud accounts
should be enabled with the one-off script below before enforcement is
deployed.
## Existing paid-account rollout
Run this as an ad-hoc Rails runner script on Chatwoot Cloud. It
intentionally targets only the Startups, Business, and Enterprise plans
and does not add `api_and_webhooks` to `manually_managed_features`, so
future billing reconciles remain authoritative.
```rb
paid_plan_names = %w[Startups Business Enterprise]
accounts = Account.where("custom_attributes ->> 'plan_name' IN (?)", paid_plan_names)
total = accounts.count
enabled = 0
skipped = 0
puts "Enabling api_and_webhooks for #{total} paid account(s)..."
accounts.find_each(batch_size: 500).with_index(1) do |account, processed|
if account.feature_enabled?('api_and_webhooks')
skipped += 1
else
account.enable_features!('api_and_webhooks')
enabled += 1
end
puts "Processed #{processed}/#{total}..." if (processed % 1000).zero?
end
puts "Done! Enabled: #{enabled}, Skipped: #{skipped}, Total: #{total}"
```
For example, save the snippet outside the repository as
`enable_api_and_webhooks.rb`, then run:
```sh
bundle exec rails runner /path/to/enable_api_and_webhooks.rb
```
## How to test
- On Cloud, use a Hacker account and confirm token-authenticated
requests to account-scoped v1 and v2 endpoints return 403, while the
same dashboard actions continue to work through session authentication.
- Confirm profile access-token controls are disabled with paid-plan copy
when all accounts are ineligible, and remain available when at least one
account has the feature.
- Confirm the Webhooks settings page shows the billing paywall for a
Cloud account without the feature; admins get the billing action and
agents get the existing ask-an-admin message.
- Confirm outgoing account webhooks stop for an ineligible Cloud account
while API-inbox webhooks still deliver.
- Confirm community and self-hosted installations retain API and webhook
behavior after upgrading, even when an existing account does not have
the stored feature bit.
### Screenshots
## Cloud
<img width="2590" height="642" alt="CleanShot 2026-07-15 at 15 13 14@2x"
src="https://github.com/user-attachments/assets/431a7bd8-1742-4e7a-b312-d3ad92015f9b"
/>
<img width="2152" height="994" alt="CleanShot 2026-07-15 at 15 14 37@2x"
src="https://github.com/user-attachments/assets/475dda48-d1c5-4be5-a3c3-7a96b9713724"
/>
---------
Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
This introduces a new `api_and_webhooks` account feature flag that will
control access to the token-authenticated API and account webhooks. The
flag is part of the Startup plan features, so paid plans — including
trials of paid plans — get it through the billing reconcile, while
accounts on the default (Hacker) plan don't, with
`manually_managed_features` available as a per-account override. The
flag defaults to enabled, and nothing enforces it yet, so this PR is
behavior-neutral — enforcement lands in a follow-up.
## What changed
- Added `api_and_webhooks` to `features.yml` (first flag on the
`feature_flags_ext_1` column, default enabled).
- Added the flag to `STARTUP_PLAN_FEATURES` in
`Enterprise::Billing::ReconcilePlanFeaturesService`, so all paid tiers
get it and the default plan loses it on reconcile.
- Added the flag to the manually manageable features list so it can be
granted per account via Super Admin.
```rb
# Enables the api_and_webhooks feature for all existing accounts and marks it
# as manually managed so cloud billing reconciles never strip it.
#
# NOT committed to source control — run manually on production.
#
# Usage:
# bundle exec rails runner enable_api_and_webhooks.rb
# ACCOUNT_ID=123 bundle exec rails runner enable_api_and_webhooks.rb
#
# Idempotent: accounts already grandfathered are skipped; safe to re-run.
probe = Internal::Accounts::InternalAttributesService.new(Account.new)
abort 'api_and_webhooks is not in valid_feature_list — deploy the feature flag PR first.' unless probe.valid_feature_list.include?('api_and_webhooks')
account_id = ENV.fetch('ACCOUNT_ID', nil)
accounts = account_id.present? ? Account.where(id: account_id) : Account.all
abort "Account with ID #{account_id} not found" if account_id.present? && accounts.empty?
total = accounts.count
puts "Grandfathering api_and_webhooks for #{total} account(s)..."
puts "Started at: #{Time.current}"
updated = 0
skipped = 0
errored = 0
accounts.find_each(batch_size: 500) do |account|
service = Internal::Accounts::InternalAttributesService.new(account)
features = service.manually_managed_features
if features.include?('api_and_webhooks') && account.feature_enabled?('api_and_webhooks')
skipped += 1
else
service.manually_managed_features = features + ['api_and_webhooks'] unless features.include?('api_and_webhooks')
account.enable_features!('api_and_webhooks')
updated += 1
end
processed = updated + skipped + errored
puts "Processed #{processed}/#{total}..." if (processed % 1000).zero?
rescue StandardError => e
errored += 1
puts "Account #{account.id}: FAILED - #{e.message}"
end
puts "Done! Updated: #{updated}, Skipped: #{skipped}, Errored: #{errored}, Total: #{total}"
```
This adds a `captain_sessions` table to log every Captain run, starting
with Assistant Responses and Copilot Responses. Each session records the
assistant, model, credits consumed, the FAQs/documents/scenario that
contributed to the response, and the full run context — giving customers
visibility into how a response was generated and giving us durable stats
on credit, FAQ, and document usage (which today only exist as ephemeral
trace metadata and an aggregate account counter).
## What changed
- New `Captain::Session` model with a `session_type` enum (`assistant`,
`copilot`). The subject (`Conversation` / `CopilotThread`) and result
(`Message` / `CopilotMessage`) classes are inferred from the session
type, so the table stores plain `subject_id` / `result_id` ids.
`result_id` is nullable so failed runs that still consumed credits can
be logged.
- Composite indexes on `[session_type, subject_id]`, `[session_type,
result_id]`, and `[account_id, session_type, created_at]` for lookup and
usage-stats queries.
- Factory and model specs.
This PR is schema + model only; the writer/instrumentation that records
sessions from the assistant and copilot flows will follow.
---------
Co-authored-by: Sony Mathew <sony@chatwoot.com>
CI runs were failing because the CircleCI Node orb's pnpm auto-install
step matches the wrong "version" field in pnpm's package.json, producing
an invalid install command since pnpm 11.12.0.
Ref: https://github.com/CircleCI-Public/node-orb/issues/262
### What changed
- Pinned `pnpm-version: 10.2.0` (matching `packageManager` in
`package.json`) on all three `node/install-pnpm` steps in
`.circleci/config.yml`, bypassing the orb's broken version
auto-detection.
Contact custom attributes filled in the pre-chat form (e.g. a CPF field)
were silently lost when the visitor's email or phone matched an existing
contact. The widget sent the attributes in a separate request that raced
the conversation create: creating the conversation merges the widget
contact into the existing contact, so the attribute update landed on the
destroyed contact and vanished without an error. New visitors were
unaffected, which made the bug hard to spot.
The fix sends contact custom attributes inside the same request that
identifies the contact. The widget conversation create endpoint now
accepts `contact.custom_attributes` and applies them through
`ContactIdentifyAction` in the same transaction as the merge, so the
submitted values always land on the surviving contact. The campaign path
folds the attributes into the existing contact update call the same way.
<details>
<summary>Reproduction script (rails runner, concurrent
threads)</summary>
```ruby
# Concurrent reproduction of the pre-chat form race that loses contact
# custom attributes when the visitor's email matches an existing contact.
#
# The old widget fired two unawaited requests on pre-chat submit. Each
# iteration replays them as real concurrent threads:
# - create thread = POST /widget/conversations: resolves the widget contact,
# then runs ContactIdentifyAction (which merges the widget contact into the
# existing contact) inside a transaction held open while the conversation
# and message are created.
# - patch thread = PATCH /widget/contact: resolves the widget contact via its
# contact inbox and applies the custom attributes through
# ContactIdentifyAction, exactly like Widget::ContactsController#update.
#
# Run with: bundle exec rails runner confirm_prechat_race.rb
# Creates throwaway contacts on Account.first and deletes them afterwards.
ITERATIONS = 20
CPF = '123.456.789-09'.freeze
account = Account.first!
inbox = account.inboxes.first!
losses = 0
ITERATIONS.times do |i|
existing = account.contacts.create!(name: 'Existing Contact', email: "race-existing-#{SecureRandom.hex(6)}@example.com")
temp = account.contacts.create!(name: 'Widget Visitor')
contact_inbox = ContactInbox.create!(contact: temp, inbox: inbox, source_id: SecureRandom.uuid)
begin
create_request = Thread.new do
ActiveRecord::Base.connection_pool.with_connection do
widget_contact = ContactInbox.find(contact_inbox.id).contact # set_contact before_action
ActiveRecord::Base.transaction do
ContactIdentifyAction.new(
contact: widget_contact,
params: { email: existing.email, phone_number: nil, name: 'Widget Visitor' },
retain_original_contact_name: true,
discard_invalid_attrs: true
).perform
sleep(0.02) # conversation + message creation keeps the transaction open
end
end
end
patch_request = Thread.new do
ActiveRecord::Base.connection_pool.with_connection do
sleep(rand * 0.02) # network jitter between the two requests
widget_contact = ContactInbox.find(contact_inbox.id).contact # set_contact before_action
ContactIdentifyAction.new(
contact: widget_contact,
params: { custom_attributes: { cpf: CPF } },
discard_invalid_attrs: true
).perform
end
end
create_request.join
patch_request.join
survivor = existing.reload
if survivor.custom_attributes['cpf'] == CPF
puts "iteration #{i + 1}: CPF survived"
else
losses += 1
puts "iteration #{i + 1}: CPF LOST (survivor custom_attributes: #{survivor.custom_attributes.inspect})"
end
ensure
account.contacts.where(id: [existing.id, temp.id]).find_each(&:destroy!)
end
end
puts
puts "#{losses}/#{ITERATIONS} iterations lost the CPF -- race #{losses.positive? ? 'confirmed' : 'not reproduced in this run'}"
```
Result: **20/20 iterations lose the CPF** (consistent across repeated
runs). The conversation create holds its transaction open across the
contact merge, so the fast attribute update either resolves the
soon-to-be-destroyed widget contact or blocks on its row lock and then
updates 0 rows — silently, with no error. This is why the customer sees
the loss every time, not intermittently.
Note: the script replays the old two-request flow at the service layer,
so it reproduces the loss even with this fix applied — the fix works by
removing the second request from the widget, not by changing the raced
code paths. The new request spec (`saves contact custom attributes on
the surviving contact when merged into an existing contact`) covers the
fixed contract.
</details>
## Closes
- Reported via support conversation:
https://app.chatwoot.com/app/accounts/1/conversations/84465
## How to reproduce
1. Create a website inbox with a pre-chat form that has a contact custom
attribute field (e.g. CPF).
2. Create a contact with a known email address.
3. As a visitor, open the widget and fill the pre-chat form using that
same email plus a value for the custom attribute.
4. Before: the attribute never appears on the contact profile. After: it
is saved on the existing contact, overwriting any stale value.
## What changed
- `POST /api/v1/widget/conversations` now permits
`contact.custom_attributes` and passes it to `ContactIdentifyAction`,
which deep-merges it atomically with the contact merge.
- The widget pre-chat form sends contact custom attributes inside the
conversation create payload instead of a separate `setCustomAttributes`
call; the campaign path includes them in the `contacts/update` payload.
Reworks the Captain assistant "hours saved" metric so it produces a
believable number. It previously multiplied assistant reply count by
customer reply-wait time, which inflated the figure into the millions of
hours. It now estimates saved time as replies × a fixed ~2 min per-reply
effort assumption. On the overview card the value renders in days once
it passes 100 hours, and the label/hint copy was updated to match.
## How to test
1. Open the Captain assistant overview page.
2. Check the "Time saved" card shows a sensible value (hours, or days
past 100h).
3. Hover the hint and confirm it reflects the per-reply estimate.
This PR adds a Captain Assistant **Overview** page to show some KPI
metrics (conversations handled, auto-resolution, handoff, hours saved,
reopen-after-resolve, conversation depth) with trend deltas vs the
previous window, a real knowledge card, and a lazily-loaded, cached LLM
welcome summary.
### Highlights
- **Two contextual banners** on the overview:
- **Inbox banner** — prompts the user to connect an inbox when the
assistant has none, so it can actually do work.
- **Coverage banner** — warns when FAQ coverage is below 85% with more
than 100 responses pending review, linking straight to the pending
queue. Dismissal persists per-assistant for 24h via localStorage.
- **Batched stats builder** (`Captain::AssistantStatsBuilder`) computes
both windows in single FILTER-aggregated scans to cut round trips,
behind new `stats`/`summary` endpoints.
- **Cards included but intentionally left dummy / not rendered yet:**
`ResponseQualityCard` (flagged responses) and `CreditUsageCard` (credit
usage + daily chart). Credits are an account-wide counter with no
per-assistant or daily history, so there is no real data to back them
yet; they ship in the codebase but are not wired into the page.
### Index migration
- Replaces `index_messages_on_sender_type_and_sender_id` with
`index_messages_on_sender_and_created` `(sender_type, sender_id,
created_at)`.
- **Why it helps:** the per-assistant windowed lookups filter `sender_*`
*and* a `created_at` range. The old 2-column index matched every
lifetime row for the assistant and filtered the time slice at the heap
(~89% of rows discarded); adding `created_at` as a range column lets
Postgres scan only the window, and fixes the row-count estimate so the
planner picks a hash join over a nested loop on `reporting_events`.
- **Why dropping the old index is safe:** the new index is a left-prefix
superset `(sender_type, sender_id, ...)`, so every query the old one
served is still served. No code references it by name, and dropping it
keeps write amplification on `messages` neutral. Built/dropped with
`CONCURRENTLY` and `if_not_exists`/`if_exists` guards.
## Preview
<img width="2572" height="1754" alt="CleanShot 2026-06-29 at 22 38
51@2x"
src="https://github.com/user-attachments/assets/3798d09e-7850-48e4-b2cd-508533f15cea"
/>
## Banners
#### Inbox connect alert
<img width="2178" height="612" alt="CleanShot 2026-06-30 at 14 26 55@2x"
src="https://github.com/user-attachments/assets/373c371c-bb7d-4291-a0f9-620673078302"
/>
#### Coverage alert
<img width="2178" height="612" alt="CleanShot 2026-06-30 at 14 25 41@2x"
src="https://github.com/user-attachments/assets/e12d6308-11b6-4ba2-88a2-8a3077dd3e8f"
/>
---------
Co-authored-by: Sivin Varghese <64252451+iamsivin@users.noreply.github.com>
Help-center onboarding jobs were getting stuck in the `generating` state
indefinitely. `Onboarding::HelpCenterArticleWriterJob` only finalized
the generation counter for two exception types
(`Firecrawl::FirecrawlError`, `ArticleBuildFailed`). Any other error
exhausted Sidekiq's default retries and landed in the dead set without
ever bumping `finished`, leaving state at `total - 1` / `generating`
until the 7-day Redis TTL expired. Observed in production: 4 generations
wedged at exactly `finished = total - 1`, four days after the run, with
no further progress — the onboarding UI showed "generating" the whole
time.
This PR adds a `discard_on StandardError` catch-all to
`HelpCenterArticleWriterJob`, after the existing `retry_on
Firecrawl::FirecrawlError` and `discard_on ArticleBuildFailed` handlers.
ActiveJob matches in declaration order, first match wins, so existing
behavior is unchanged — the catch-all only absorbs errors that
previously fell through to unhandled retry-then-dead-set. It routes
through the same `on_writer_failure` → `finalize` path, so state always
progresses to `completed`.
<details><summary>
##### Script to remove dead jobs
</summary>
<p>
```rb
# One-off: unstick Onboarding::HelpCenterGenerationState keys that are wedged in
# "generating" because a writer job died on an unhandled exception (neither
# FirecrawlError nor ArticleBuildFailed) and exhausted Sidekiq retries without
# ever calling record_article_finished.
#
# These portals have real articles (finished is 1 short of total). Marking them
# "completed" is the honest terminal state: generation is done, one article failed.
#
# Run on a prod box (dry-run first, then REMOVE_DRY_RUN=1):
# RAILS_ENV=production bundle exec rails runner scripts/help_center_investigation/unstick_generating.rb
#
# To actually write, set REMOVE_DRY_RUN=1 in the environment.
pattern = format(Redis::Alfred::HELP_CENTER_GENERATION, id: '*')
dry_run = ENV['REMOVE_DRY_RUN'].blank?
stuck = []
Redis::Alfred.with do |conn|
conn.scan_each(match: pattern, count: 1000) do |key|
h = conn.hgetall(key)
next unless h['status'] == 'generating'
gen_id = key.sub('HELP_CENTER_GENERATION::', '')
stuck << { gen_id: gen_id, total: h['total'], finished: h['finished'], key: key }
unless dry_run
conn.hset(key, 'status', 'completed')
conn.expire(key, Onboarding::HelpCenterGenerationState::TTL)
end
end
end
puts "#{dry_run ? '[DRY RUN] ' : ''}Found #{stuck.size} stuck 'generating' states:"
stuck.each do |s|
puts " gen=#{s[:gen_id]} total=#{s[:total]} finished=#{s[:finished]} -> #{dry_run ? 'would mark completed' : 'marked completed'}"
end
puts
puts 'Re-run with REMOVE_DRY_RUN=1 to apply.' if dry_run
````
</p>
</details>
Outbound voice calls were being auto-assigned to the wrong agent. When
an agent placed an outbound call, the conversation was created without
an assignee, so inboxes with auto-assignment enabled would round-robin
it to a different agent instead of keeping it with the person who
actually made the call. This made it hard to tell which agent was on an
active call.
## What changed
- Set the calling agent as the conversation's assignee when creating an
outbound voice call conversation.
- This prevents the generic auto-assignment handler from treating the
conversation as unassigned and reassigning it.
- Added specs covering the assignment, including a regression case with
inbox auto-assignment enabled.
**Note:** this applies to newly placed calls; it does not retroactively
fix conversations that were already mis-assigned.
---------
Co-authored-by: Tanmay Deep Sharma <tanmaydeepsharma21@gmail.com>
Co-authored-by: Tanmay Deep Sharma <32020192+tds-1@users.noreply.github.com>
**Broaden the Firecrawl `map` search term list so help-center onboarding
doesn't skip sites with non-standard docs paths**
Help-center onboarding was skipping ~70% of new accounts, and ~60% of
those skips came from a single failure: `"map returned no links"`. The
root cause was an overly narrow hardcoded search query passed to
Firecrawl's `map` endpoint.
The curator (`Onboarding::HelpCenterCurator`) calls `Firecrawl.map(url,
search: MAP_SEARCH)` to discover candidate pages before the LLM curation
step. `MAP_SEARCH` was hardcoded to `"docs help support faq"` — a 4-term
list that only matched sites whose help content sat at `/docs`, `/help`,
`/support`, or `/faq`. Sites using `/resources`, `/guides`, `/kb`,
`/articles`, `/handbook`, `/learn`, `/how-to`, `/tutorial`,
`/troubleshooting`, or a docs subdomain found nothing, so the job raised
`CurationSkipped` and left the portal empty.
Firecrawl's `search` param is a grep-style substring filter across URL,
title, and description (not a semantic query), so the fix is to broaden
the term list rather than drop it. The LLM curator downstream
(`Captain::Llm::HelpCenterCurationService`) already filters returned
links by quality — it has the full URL-path-priority prompt and a
25-article hard ceiling — so a wider crawl net is safe and doesn't
change the final article quality bar.
**What changed**
- `enterprise/app/services/onboarding/help_center_curator.rb`:
`MAP_SEARCH` broadened from `"docs help support faq"` to a 13-term list
covering common help-content path hints. Comment added documenting why
the term list exists and that the LLM curator does the real filtering.
LeadSquared sync now recovers automatically when a contact's cached lead
has been deleted or merged on the LeadSquared side. Previously the stale
lead id was never cleared, so every new conversation or contact update
for that contact failed with "Lead not found"
(`MXInvalidEntityReferenceException`) indefinitely.
## What changed
- Activity sync: on a "Lead not found" error while posting a
conversation/transcript activity, clear the cached `leadsquared_id`,
re-resolve the contact to a fresh lead, and retry the activity once
(guarded against loops and duplicate leads).
- Contact sync: on the same error while updating an existing lead, clear
the cached id and create a fresh lead instead.
- Fix `get_lead_id` to actually return early for unidentifiable contacts
(the guard previously fell through).
## How to reproduce
1. For a LeadSquared-enabled account, point a contact's cached lead id
at a lead that no longer exists in LeadSquared.
2. Update the contact, or create/resolve a conversation for it.
3. Before: the sync fails repeatedly with "Lead not found" and never
self-corrects. After: the stale id is cleared, a fresh lead is
resolved/created, and subsequent syncs reuse the healed id.
---------
Co-authored-by: Tanmay Deep Sharma <32020192+tds-1@users.noreply.github.com>
After entering their account details, new admins land on an **Inbox
setup** screen that shows what we've already set up for them and lets
them connect their conversation channels without leaving onboarding. It
surfaces the auto-created live chat widget (and Help Center on
Enterprise), highlights channels detected from their website, and offers
a **View all** dialog to connect any supported channel inline.
### Channel status
| Channel | How it connects | Status | PR |
|---|---|---|---|
| Live chat (Website) | Auto-created during setup | ✅ Done |
https://github.com/chatwoot/chatwoot/pull/14314 |
| WhatsApp | Meta embedded signup | ✅ Done |
https://github.com/chatwoot/chatwoot/pull/14619 |
| Facebook | Login + page picker | ✅ Done |
https://github.com/chatwoot/chatwoot/pull/14619 |
| Instagram | OAuth redirect | ✅ Done |
https://github.com/chatwoot/chatwoot/pull/14568 |
| TikTok | OAuth redirect | ✅ Done |
https://github.com/chatwoot/chatwoot/pull/14569 |
| LINE | Inline credential form | ✅ Done | — |
| Telegram | Inline credential form | ✅ Done | — |
| Gmail / Outlook | OAuth (email) | ⚠️ Disabled — coming in a follow-up
| https://github.com/chatwoot/chatwoot/pull/14567 |
| SMS / API / Voice / Other email | — | ⛔ Unavailable | — |
---------
Co-authored-by: Sivin Varghese <64252451+iamsivin@users.noreply.github.com>
When an inbox has **"Allow messages after conversation is resolved"**
disabled, the live-chat widget should hide the reply box once a
conversation is resolved — but users could still send a reply, silently
reopening it. This syncs the widget's conversation status on reconnect
so the reply box hides correctly.
## Closes
- CW-7272
## How to reproduce
1. Disable **Allow messages after conversation is resolved** on an
inbox.
2. Open the widget, then let its websocket drop (background tab / lose
network).
3. While disconnected, get the conversation resolved (e.g. auto-resolve
on inactivity).
4. Reconnect → reply box is still visible and a sent message reopens the
resolved conversation.
## Why
Reply-box hiding is gated on the widget's local status, updated via the
`conversation.status_changed` socket event. If the resolve happens while
disconnected, the event is missed — and on reconnect the widget only
re-synced messages, never the status, so it stayed stale at `open`.
## What changed
- `onReconnect` now also dispatches
`conversationAttributes/getAttributes` to refresh status after a missed
event.
- Added a spec covering the reconnect behavior.
Co-authored-by: Sivin Varghese <64252451+iamsivin@users.noreply.github.com>
This PR adds a reload-safe onboarding Help Center generation status
path. Previously, generation progress was pushed through ActionCable,
which meant the onboarding UI could lose context after a page reload or
missed websocket event. The new endpoint exposes the current generation
id, raw Redis generation state, and Help Center article/category counts
so clients can recover state by fetching from the backend.
**What changed**
- Added an onboarding Help Center generation status endpoint.
- Persisted `help_center_generation_id` when generation is enqueued.
- Removed Help Center generation ActionCable broadcasts completely.
- Kept generation progress in Redis as the backend source of truth.
- Kept Help Center generation out of the onboarding hot path; the
existing onboarding controller does not start generation yet.
**How to test**
1. Start Help Center generation for an account.
2. Fetch the onboarding generation status endpoint and verify it returns
the generation id, Redis state, and article/category counts.
3. Reload the onboarding UI or client state and fetch again to confirm
progress can be recovered without relying on websocket events.
4. Verify skipped/completed generation states are reflected from Redis.
## Related PRs
- https://github.com/chatwoot/chatwoot/pull/14569
- https://github.com/chatwoot/chatwoot/pull/14568
- https://github.com/chatwoot/chatwoot/pull/14567
- https://github.com/chatwoot/chatwoot/pull/14619
- https://github.com/chatwoot/chatwoot/pull/14565 (Primary onboarding
PR)
Captain custom tools previously used their own hand-rolled `Net::HTTP`
request code. This moves them onto `SafeFetch`, the same shared
HTTP-fetching helper already used by webhooks, uploads, avatar imports,
and the Captain page crawler. Behavior for normal tools is unchanged —
they just now share one consistent path for host resolution, timeouts,
response size limits, and redirect handling.
**What changed**
- `HttpTool#execute_http_request` now delegates to `SafeFetch.fetch`
instead of building `Net::HTTP` requests by hand. Auth headers, basic
auth, metadata headers, JSON content-type, and the 1 MB response cap all
map onto `SafeFetch` options.
- Removed ~80 lines of bespoke request/validation plumbing from
`HttpTool`.
- The custom tools `test` endpoint now reads the response body string
directly (the executor returns the body rather than a response object).
**How to test**
1. Enable `custom_tools` (or `captain_integration_v2`) for an account.
2. Create a Captain custom tool pointing at a public HTTPS endpoint
(e.g. a test API).
3. Use the **Test** button in the tool form — you should get a success
result.
4. Run the tool from a Captain conversation and confirm the response is
returned/templated as before.
**Gotchas**
- **Local dev:** `SafeFetch` blocks requests to private/loopback
addresses by default. If you're testing a custom tool against a service
on `localhost` or a private IP during development, set
`SAFE_FETCH_ALLOW_PRIVATE_NETWORK=true` or the request will be rejected.
(This matches how the rest of `SafeFetch` already behaves locally.)
- **Test endpoint status field:** on success the `test` response now
reports `status: 200` rather than the exact 2xx code (201/204/etc.),
because `SafeFetch` signals success-vs-failure rather than exposing the
raw response. The UI only checks the 2xx range, so this is invisible
there — but worth knowing if anything consumes the API directly.
- **Non-2xx responses** still surface as an error (same as before), now
via `SafeFetch::HttpError`.
Extracts the Meta JS SDK logic for the WhatsApp embedded-signup and
Facebook Page connect flows out of their settings components into two
reusable composables. No behavior change — the settings inbox-creation
flows work exactly as before; this just makes the SDK orchestration
reusable (an upcoming onboarding PR consumes them) and adds unit
coverage.
## What changed
- `useWhatsappEmbeddedSignup` — owns the embedded-signup popup (SDK
load, FB.login, and the auth-code / postMessage race).
`WhatsappEmbeddedSignup.vue` now consumes it.
- `useFacebookPageConnect` — owns FB.login (page scopes) + page fetch,
with SDK preload split from login so the popup opens within the click's
activation window. `Facebook.vue` now consumes it.
- Adds unit specs for both composables.
## Related PRs
- https://github.com/chatwoot/chatwoot/pull/14569
- https://github.com/chatwoot/chatwoot/pull/14568
- https://github.com/chatwoot/chatwoot/pull/14567
- https://github.com/chatwoot/chatwoot/pull/14649
- https://github.com/chatwoot/chatwoot/pull/14565 (Primary onboarding
PR)
Instagram webhooks use a Redis mutex to protect the first-message path
for a contact/account pair. When multiple webhook events for the same
Instagram contact arrive at nearly the same time, they can all enter the
“find or create” flow together.
There are two pieces involved:
`ContactInbox` maps the external Instagram scoped user id to a Chatwoot
contact inside an inbox. That side already has a unique `(inbox_id,
source_id)` index and retry handling, so duplicate contact-inbox
creation is mostly protected at the database layer.
`Conversation` creation is more fragile. The message builder looks for
an existing active conversation for the contact/inbox and creates one
when none is found. If two first-message jobs run concurrently, both can
see “no active conversation yet” and both can create a conversation. The
mutex exists to bump those jobs apart long enough for the first one to
create the conversation, so the next one appends to it instead of
creating a duplicate.
In production, the old behavior could turn that race dampener into a
drop path. Once `Webhooks::InstagramEventsJob` exhausted its lock
retries, ActiveJob stopped retrying with
`MutexApplicationJob::LockAcquisitionError`. Since ActiveJob retries are
not FIFO, later jobs could still win the lock while older jobs kept
getting deferred until exhaustion.
## Mutex Application Job Changes
This adds `retry_on_lock_conflict` as a small wrapper around the
existing `retry_on LockAcquisitionError` pattern.
The new API keeps the default behavior intact, but lets jobs explicitly
define what should happen after lock retry exhaustion:
```ruby
retry_on_lock_conflict wait: 1.second,
attempts: 3,
on_exhaustion: :process_without_lock
```
The fallback receives the original job arguments, so job classes do not
need to reach into ActiveJob internals like `arguments.first`.
## Instagram Fallback
Instagram now processes the webhook payload even after the mutex retry
window is exhausted.
This matches what the mutex was meant to do in the first place: bump
concurrent events apart long enough to avoid duplicate conversation
creation, not permanently block or drop webhook events. If a job cannot
acquire the lock after a few retries, it continues through the normal
Instagram processing path without the mutex.
## Retry And Lock Tuning
The Instagram lock retry window now uses deterministic backoff:
```ruby
retry_on_lock_conflict wait: ->(executions) { executions.seconds },
attempts: 3,
on_exhaustion: :process_without_lock
```
The lock TTL is also set explicitly:
```ruby
with_lock(key, 3.seconds)
```
This matters because Rails treats `attempts` as total executions, not
retries after the first execution. With a fixed `wait: 1.second` and
`attempts: 3`, the exhaustion handler can run roughly two seconds after
the first lock conflict. That is earlier than a 3-second lock TTL, so
the fallback could process without the lock while the original mutex is
still valid. That would reopen the duplicate-conversation race the lock
is meant to dampen.
Using a proc wait makes the timing predictable and avoids Rails jitter
for this retry path. The first conflict waits about 1 second, the second
waits about 2 seconds, and the final attempt happens around the 3-second
mark. That lines the retry window up with the Redis lock TTL before
falling back to `process_without_lock`.
The protected race window is intentionally small. `ContactInbox`
creation already has database uniqueness protection, while conversation
creation is the softer `find active conversation || create` path. We
only need to give the first job enough time to create the conversation
state that later jobs should reuse.
The mutex still does not preserve message order, and ActiveJob retries
are not FIFO. A longer retry window would mostly add latency for hot
Instagram contacts without making ordering more correct. After the lock
TTL has elapsed, processing without the lock is the better tradeoff than
dead-lettering customer messages.
Upgrades the frontend toolchain to Vite 6 and tidies up the build config
along the way. Behavior is unchanged for end users; this is dev/build
infra.
## What changed
- `vite` 5.4 → 6.4, `@vitejs/plugin-vue` → 5.2, `vite-plugin-ruby` → 5.2
(with matching `vite_rails`/`vite_ruby` gem bumps).
- Dropped the `vite-node` 2.0.1 pnpm override — no longer needed now
that vitest 3 runs on Vite 6 directly.
- Split the single `vite.config.ts` into:
- `vite.config.ts` (app), `vite.lib.config.ts` (SDK), `vite.shared.ts`
(aliases / Vue options), `vitest.config.ts` (tests).
- `pnpm build:sdk` now selects the SDK config explicitly instead of
branching on `BUILD_MODE=library`. SDK output path is unchanged
(`public/packs/js/sdk.js`).
No changes needed to Docker images, deployment scripts, or CI — Node 24
and pnpm 10 are already past Vite 6's floor, and the rake
`assets:precompile` hook still drives the SDK build via `pnpm`.
## How to test
- `pnpm dev` and verify the dashboard, widget, and survey routes load
and HMR works.
- Load a Chatwoot site widget on a test page and confirm `sdk.js` is
served and the widget mounts.
- `RAILS_ENV=production bundle exec rake assets:precompile` and confirm
`public/packs/js/sdk.js` plus the rest of the manifest are produced.
- `pnpm test` for the JS suite.
---------
Co-authored-by: Sivin Varghese <64252451+iamsivin@users.noreply.github.com>
Co-authored-by: Sony Mathew <2040199+sony-mathew@users.noreply.github.com>
When connecting an Instagram inbox during onboarding, the OAuth flow
used to drop users in inbox settings, breaking onboarding. The OAuth
start endpoint now accepts an optional `return_to=onboarding` hint,
carried tamper-proof inside the signed state (a claim on the Instagram
JWT), and the callback uses it to return the user to the onboarding
inbox-setup screen. Without the hint, behavior is unchanged.
This is the backend half only; the frontend that sends
`return_to=onboarding` ships separately.
Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
When connecting a Gmail or Outlook inbox during onboarding, the OAuth
flow used to drop users in inbox settings, breaking onboarding. The
OAuth start endpoint now accepts an optional `return_to=onboarding`
hint, carried tamper-proof inside the signed `state`, and the callback
uses it to return the user to the onboarding inbox-setup screen. Without
the hint, behavior is unchanged.
This is the backend half only; the frontend that sends
`return_to=onboarding` ships separately.
Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
Outgoing email on a Microsoft email inbox was failing with `535 5.7.3
Authentication unsuccessful` immediately after the user
re-authenticated, while incoming (IMAP) continued to work. The root
cause is in our OAuth callback: we persist the id_token's `email` claim
into `channel.imap_login`, but Microsoft's SMTP AUTH (XOAUTH2) validates
the username against the access token's **UPN**, not the mailbox's
primary SMTP address or aliases. When those diverge — common in tenants
that use one domain for sign-in identities and another for mailbox
addresses — SMTP rejects every send.
This PR makes `OauthCallbackController#update_channel` prefer
`preferred_username` (v2.0) / `upn` (v1.0) from the id_token over
`email`, with `email` as the fallback so Google flows are unchanged.
## What changed
- `app/controllers/oauth_callback_controller.rb` — extract
`imap_login_identity` (default: `users_data['email']`, same as before).
`update_channel` now calls it instead of inlining the email claim. No
behavioural change in the base controller.
- `app/controllers/microsoft/callbacks_controller.rb` — override
`imap_login_identity` to return `preferred_username || upn || super`.
Provider-specific knowledge stays in the provider subclass; Google's
flow is literally untouched.
- `spec/controllers/microsoft/callbacks_controller_spec.rb` — adds one
example that reproduces the divergent shape (`email: <alias>`,
`preferred_username: <upn>`) and asserts `imap_login` lands on the UPN
while `channel.email` stays on the mailbox alias.
`channel.email` and `find_channel_by_email` still key on the id_token's
`email` claim everywhere, so customer-facing From identity and
reconnect-by-email matching are unchanged.
Behavioural matrix:
| Provider / shape | `imap_login` before | `imap_login` after |
|-----------------------------------------------------------------|---------------------|--------------------|
| Microsoft, `upn == email` (common case) | email | UPN (= email, same
string) |
| Microsoft, `upn != email` (e.g. UPN on one domain, mailbox alias on
another) | alias (broken) | UPN (works) |
| Google | email | email (no change, base impl used) |
## Why this happens (Microsoft side, for context)
Two facts that together produce the bug — one is documented, the other
we verified empirically because Microsoft's docs don't address it:
1. **`email` and `upn` are different claims and can legitimately
diverge.** In Entra, the UPN is the sign-in identity; the id_token's
`email` claim is the user's mailbox property (which can be a proxy
address). For v2.0 tokens (which is what we use —
`/common/oauth2/v2.0/token` in `MicrosoftConcern`), the documented
"username to sign in as" claim is `preferred_username`. See [ID token
claims
reference](https://learn.microsoft.com/en-us/entra/identity-platform/id-token-claims-reference).
So `users_data['email']` was the wrong source for `imap_login`; tenants
where UPN == primary SMTP happened to mask the bug.
2. **Exchange's XOAUTH2 SMTP rejects aliases in the `user=` field, even
though IMAP accepts them.** This asymmetry is **not** in [Microsoft's
canonical XOAUTH2
doc](https://learn.microsoft.com/en-us/exchange/client-developer/legacy-protocols/how-to-authenticate-an-imap-pop-smtp-application-by-using-oauth)
— the doc treats the SASL `user=` field uniformly across IMAP, POP, and
SMTP, with only a shared-mailbox carve-out spelled out. We confirmed the
SMTP-strict behaviour empirically by running an XOAUTH2 AUTH probe with
the same access token: `user=<alias>` returned `535 5.7.3`, `user=<UPN>`
returned `235`. Same token, only the `user=` field differed. That's what
tied the symptom to claim-shape.
Hence the patch: read the documented "sign-in name" claim and persist
that, not the customer-facing mailbox address.
## How to reproduce (before the fix)
1. Set up a Microsoft email inbox in a tenant where the user's UPN
domain differs from the user's primary SMTP / mailbox domain (e.g. UPN
`user@tenant-a.example`, mailbox `User@tenant-b.example` where
`tenant-b.example` is a proxy address on the same mailbox).
2. Connect or re-authenticate the inbox through the standard flow.
3. Inspect the channel:
```ruby
ch.imap_login # => "User@tenant-b.example" (alias — wrong)
token = ch.provider_config['access_token']
JSON.parse(Base64.urlsafe_decode64(token.split('.')[1].then { |s| s +
'=' * (-s.length % 4) }))['upn']
# => "user@tenant-a.example" (UPN — what SMTP actually needs)
```
4. Send a reply. SMTP returns `535 5.7.3 Authentication unsuccessful`.
Incoming IMAP continues to work fine.
After the fix, `imap_login` is set to the UPN at callback time and SMTP
succeeds.
## Notes for review
- The id_token decoding path (`users_data`) already exists and is
trusted by the rest of the callback — no new attack surface.
- Scoped to Microsoft on purpose. A provider-agnostic fallback chain in
the base controller would also have worked (Google id_tokens don't carry
`preferred_username` / `upn`, so it'd land on email anyway), but keeping
Google's code path identical to today removes any risk of an unintended
interaction with whatever a future Google update might add to its
id_token. Pattern matches the existing `find_channel_by_email` override
Google already has.
- The [id_token claims
reference](https://learn.microsoft.com/en-us/entra/identity-platform/id-token-claims-reference)
warns that `preferred_username` is mutable and "can't be used to make
authorization decisions." That warning targets apps using the claim as a
stable cross-session identifier for app-level authz — we're not. We use
it as the SASL `user=` string for SMTP AUTH, validated by Microsoft
against the same token it just issued. The claim's mutability matters
only if a tenant admin renames a UPN between re-auths, in which case the
stored `imap_login` goes stale and SMTP 535s — but the pre-patch code
has the identical mutability characteristic on the `email` claim (rename
a mailbox's primary SMTP and you get the same stale-then-535). The patch
doesn't enlarge that failure surface; it shrinks it, because
UPN-vs-alias divergence is now handled rather than always-broken.
- Related but out of scope: SMTP failures don't trigger
`channel.authorization_error!` today. `ExceptionList::SMTP_EXCEPTIONS`
(`lib/exception_list.rb`) only contains `Net::SMTPSyntaxError`;
`Net::SMTPAuthenticationError` bubbles unhandled, and
`ApplicationMailer#handle_smtp_exceptions` only logs. So a 535 — whether
from this bug or any other identity drift — never prompts re-auth in the
UI, unlike the IMAP path (`fetch_imap_emails_job` catches
`OAuth2::Error` and calls `authorization_error!`). Worth a separate PR;
flagging here so we don't pretend stale-identity SMTP errors are
self-healing.
- The alternative I considered — decoding the access token's `upn`
directly — is more correct in principle but relies on Microsoft access
tokens being JWTs, which is undocumented behaviour for the
`outlook.office.com` audience and contradicts the [docs
guidance](https://learn.microsoft.com/en-us/entra/identity-platform/access-tokens)
that access tokens are opaque to clients.
- **Not addressed here, flagging for follow-up:** existing channels that
were re-authenticated before this fix still hold the alias in
`imap_login` and will keep 535ing until the user re-auths again. A
one-shot rake task could decode the stored access tokens and reconcile,
but fixing forward via natural re-auth is lower-risk; let me know if you
want the backfill.
- Also out of scope: `app/mailers/conversation_reply_mailer_helper.rb`
reads `provider_config['access_token']` raw without going through
`Microsoft::RefreshOauthTokenService`, while the IMAP path refreshes.
That's a real asymmetry but unrelated to this bug (the token here was
minutes-old) and worth its own PR.
When connecting a TikTok inbox during onboarding, the OAuth flow used to
drop users in inbox settings, breaking onboarding. The OAuth start
endpoint now accepts an optional `return_to=onboarding` hint, carried
tamper-proof inside the signed `state` (a claim on TikTok's signed JWT),
and the callback uses it to return the user to the onboarding
inbox-setup screen. Without the hint, behavior is unchanged.
This is the backend half only; the frontend that sends
`return_to=onboarding` ships separately.
## What changed
- `Tiktok::IntegrationHelper`: the signed JWT carries an optional
`return_to` claim, added only when present (a request without it is
byte-identical to before); added `tiktok_token_return_to` to read it;
`decode_token` now returns the full payload and `verify_tiktok_token`
derives the account id from it.
- `Tiktok::AuthorizationsController#create` passes `params[:return_to]`
into the token.
- `Tiktok::CallbacksController` redirects to the onboarding inbox-setup
screen when `return_to == 'onboarding'`, before the normal
settings/agents redirect.
- Added the `app_onboarding_inbox_setup` route (shared with the sibling
Gmail/Outlook and Instagram PRs — keep a single copy on merge to avoid a
duplicate route name).
Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
Depends on: https://github.com/chatwoot/chatwoot/pull/14370
This PR creates a new onboarding controller, this allows more control
that the default account update API. Allowing us to spin tasks and
update details required specifically during the onboarding flow
During account enrichment, WebsiteBrandingService now probes the signup
domain's MX records to infer whether email is hosted on Google Workspace
or Microsoft 365, adding `email_provider` (`google`/`microsoft`/`nil`)
to `brand_info`. Matching is anchored on a label boundary so lookalike
domains aren't misclassified, and lookup failures fall back to `nil`.
This lets downstream UI suggest the right mailbox integration (Gmail vs
Outlook).
This PR updates two dependencies — `faraday` (2.14.1 → 2.14.2) and `jwt`
(2.10.1 → 2.10.3) — to pick up security patches flagged by
`bundle-audit`. Both are bumped to the minimal patched release within
their existing major lines to keep the blast radius small.
### Faraday
`Faraday::Connection#build_exclusive_url` still allowed a
protocol-relative host override when the request target was passed as a
`URI` object (rather than a `String`), bypassing the earlier fix for the
string-based variant (CVE-2026-25765 / GHSA-33mh-2634-fwr2). On a
fixed-base connection this could redirect a request to an
attacker-controlled host while still forwarding connection-scoped
headers such as `Authorization` — i.e. off-host request forgery
(CVE-2026-33637 / GHSA-5rv5-xj5j-3484).
The fix is a clean patch bump to `2.14.2`, within Faraday's existing
version range — no API changes and no other gems affected.
### JWT
`jwt` 2.10.1 accepts an empty/`nil` HMAC key during verification:
`JWT.decode(token, "", true, algorithm: 'HS256')` (and keyfinder paths
returning `""`/`nil`) verify a forged token, because the empty-key HMAC
digest is treated as valid and `enforce_hmac_key_length` defaults to
`false` (CVE-2026-45363, High).
The advisory offers two fixes — `~> 2.10.3` or `>= 3.2.0`. We chose
**2.10.3** deliberately: jumping to 3.x cascaded into upgrading
`oauth2`, `twilio-ruby`, `googleauth`, `web-push`, and `signet` (all
pinned `jwt < 3.0`), and `jwt` is used directly in 8+ places here (token
services, OAuth callbacks, integration helpers), so a major bump carries
real breakage risk for no extra security benefit. The Gemfile is pinned
`'~> 2.10', '>= 2.10.3'` to hold the 2.x line.
**Spec changes.** 2.10.3 tightens key handling: HMAC sign/verify now
raises on a `nil`, empty, or non-`String` key instead of silently
coercing it. A few specs relied on the old lax behaviour and needed
updating:
- `microsoft` / `google` callback specs built unsigned ID tokens via
`JWT.encode(payload, false)`. Replaced with the correct unsigned form,
`JWT.encode(payload, nil, 'none')`.
- `instagram` / `linear` / `shopify` helper specs have a "client secret
not configured" context where `client_secret` is `nil`. Their shared
`valid_token` `let` signed with that `nil` secret, which Ruby evaluates
before the helper runs — now raising. Since the helper short-circuits on
the blank secret and never decodes the token, those contexts now
override `valid_token` with a throwaway string.
**Production is unaffected.** Every production HMAC path uses a real,
non-empty key — `Rails.application.secret_key_base` (`BaseTokenService`,
`Widget::TokenService`) or a client secret guarded by `return if
client_secret.blank?` (Instagram/TikTok/Shopify/Linear helpers). The one
`nil`-key call, `JWT.decode(id_token, nil, false)` in
`OauthCallbackController`, runs with verification disabled, so the key
is never inspected. Twilio voice tokens use `Twilio::JWT::AccessToken`
from `twilio-ruby`, not this gem. The specs failed precisely because
they exercised the unsafe empty-key pattern the patch now blocks —
production never did.
On high-volume accounts, the dashboard sidebar's conversation count
badges fall behind because a meaningful share of
`/api/v1/accounts/:id/conversations/meta` requests get rate-limited
(per-user throttle, default 30 req/min).
Root cause is in `conversationStats.js`. The tiered debounce uses
`allCount` from the last response to pick a wait interval. `allCount`
reflects the user's *current filtered scope*, not the account's true
volume — so an agent viewing a small filter on a busy account falls into
the most aggressive tier (500ms wait / 1.5s maxWait → up to 40
calls/min/tab) and trips the throttle.
## What changed
`app/javascript/dashboard/store/modules/conversationStats.js`:
- Short-tier `maxWait`: `1500 → 2000` (caps short-tier at 30/min/tab
instead of 40)
- Super-long-tier threshold: `allCount > 5000 → > 2000` (more
high-volume accounts fall into the safe 3/min/tab tier)
- Middle-tier threshold unchanged (`> 100`)
| Tier (allCount) | wait / maxWait | Calls/min/tab |
|---|---|---|
| `> 2000` | 10s / 20s | 3 |
| `> 100` | 5s / 10s | 6 |
| else | 500ms / 2s | 30 |
## Trade-off
Badge updates (including those triggered by the agent's own action) may
lag by up to the tier's `maxWait` — worst case 20s for accounts with >
2000 open conversations in the active scope. The conversation list
itself and push notifications continue to update in real time; only the
numeric badge is debounced.
## Not in scope
- Sticky-max `allCount` to fix the underlying tier-selection signal —
defer until the simpler tuning is validated in production
- Optimistic count updates on local user actions — adds non-trivial
state management for a cosmetic lag
This PR makes SAML login independent of Rails session cookies
## Problem
The normal SAML login flow should be straightforward:
- User opens Chatwoot.
- Chatwoot creates `_chatwoot_session`.
- User starts SSO.
- Chatwoot redirects the browser to the SAML provider.
- The provider authenticates the user.
- The provider sends the browser back to Chatwoot's ACS URL.
- Chatwoot reads the SAML response, finds or creates the user, and logs
them in.
The fragile step is the ACS callback. Most SSO flows return to the app
through browser redirects where cookies usually pass through as
expected. **ADFS commonly returns the SAML response with a cross-site
POST**. With Chatwoot's session cookie using `SameSite=Lax`, browsers
may not send `_chatwoot_session` on that POST.
SAML validation itself does not need the old Rails session cookie. The
problem was our callback handoff after validation. DeviseTokenAuth
stores the verified OmniAuth payload in Rails session, then redirects to
a second callback route. If the browser does not preserve that session,
Chatwoot has already received a valid SAML response but can no longer
finish login.
## Solution
This PR removes the session-backed handoff for SAML only:
- The SAML callback completes login in the same request where OmniAuth
validates the SAML response.
- Chatwoot reads the verified auth payload directly from
`request.env['omniauth.auth']`.
- Account context and RelayState come from callback params or OmniAuth
env data, not Rails session.
- Other OmniAuth providers continue using the existing DeviseTokenAuth
flow.
- Mobile SAML still works when the IdP returns `RelayState=mobile`; the
callback redirects to the mobile deep link with the generated SSO token.
The previous SAML override used `303 See Other` to avoid replaying the
SAML POST into the second callback route. This change keeps that intent,
but removes the second callback route for SAML entirely.
## Screen recording
### SP Initiated
https://github.com/user-attachments/assets/b0735e93-3864-4cc3-b6fc-419fff4b549e
### IDP Initiated
https://github.com/user-attachments/assets/3ded0246-933c-4c85-9b7c-fa15fdc34883
## Testing
Manual validation:
- Complete a SAML login.
- In the browser network trace, find the IdP POST to
`/omniauth/saml/callback?account_id=<account-id>`.
- Confirm it redirects directly to `/app/login?...sso_auth_token=...`
for web login.
- For mobile, confirm `RelayState=mobile` redirects to the configured
mobile deep link.
- Confirm there is no intermediate `/auth/saml/callback` request.
Testing with mocksaml.com:
- Configure Chatwoot with a public `FRONTEND_URL`.
- Set the mocksaml ACS URL to:
```text
https://<chatwoot-host>/omniauth/saml/callback?account_id=<account-id>
```
- Set the mocksaml audience/SP entity ID to the value shown in Chatwoot
SAML settings, usually:
```text
https://<chatwoot-host>/saml/sp/<account-id>
```
- Use an email returned by mocksaml that exists in the SAML-enabled
account.
- Start login from Chatwoot's SSO login page.
- Confirm the callback redirects directly to the app login URL with an
SSO token.
---------
Co-authored-by: Sojan Jose <sojan@pepalo.com>
Quoted email replies is now available to every account by default.
Previously this was gated behind the `quoted_email_reply` account-level
feature flag, so accounts needed it toggled on (via Super Admin) before
agents saw the toggle in the reply box.
## How to test
1. Open any conversation on an email inbox.
2. Confirm the **Quote previous email** toggle is visible in the reply
box (and is **not** visible on private notes or non-email channels).
3. Toggle it on, type a reply, and send — the outbound email should
include the quoted prior email below your message.
4. Toggle it off and send another reply — the quoted block should not
appear.
5. The toggle preference should persist per channel type (UI setting),
as before.
6. Verify the toggle works on a brand new account with no feature flags
flipped on (previously it would have been hidden).
## What changed
- Removed all `isFeatureEnabledonAccount(..., QUOTED_EMAIL_REPLY)` gates
from `ReplyBox.vue`, so the toggle and quoted-content behavior are
unconditional on email channels.
- Removed the `QUOTED_EMAIL_REPLY` constant from
`dashboard/featureFlags.js`.
- Marked the flag as `deprecated: true` in `config/features.yml` (kept
the entry in place to preserve FlagShihTzu bit positions on existing
accounts; `deprecated: true` hides it from the Super Admin UI).
- Dropped the now-unnecessary
`account.enable_features('quoted_email_reply')` setup from the message
builder spec.
`spec/lib/safe_fetch_spec.rb` has been flaking intermittently under
full-suite runs with errors like:
```
expected SafeFetch::FileTooLargeError, got #<SafeFetch::FileTooLargeError: exceeded 1048576 bytes>
```
The class name on both sides is identical — yet RSpec reports a
mismatch. This PR replaces the constant-identity assertions in this spec
with a name-based matcher so the comparison stops depending on the live
Class object's identity. We have made a similar fix earlier, but that
wasn't addressing the core of the issue in #14139.
**How to reproduce:** The flake only surfaces under load. Running the
spec in isolation almost always passes, here's a [run failing in
CI](https://github.com/chatwoot/chatwoot/actions/runs/25852294516/job/75961520248?pr=14370)
## What's actually going on
Three facts combine:
1. **Test env reloads classes.** `config/environments/test.rb` sets
`cache_classes = false` so Zeitwerk reloads autoloadable code on demand.
2. **`lib/` is on the reloadable autoload tree.**
`config/application.rb` adds `lib` to `eager_load_paths`, which (with
`eager_load = false` in test) makes it lazily loaded by Zeitwerk's
*main* (reloading) autoloader. `lib/safe_fetch/` lives under that
umbrella.
3. **RSpec's `raise_error(Klass)` snapshots the Class object.**
`raise_error` matcher captures `Klass` (a specific `Class` instance)
when the matcher is built. At raise time it compares with `Module#===`,
which is identity-based.
When the executor between examples triggers
`Rails.application.reloader.reload!`, Zeitwerk does
`remove_const(:SafeFetch)` and re-installs an autoload trigger. The next
access produces a **fresh** `Class` object for
`SafeFetch::FileTooLargeError` — same name, different identity. The
matcher's snapshot now points at the dead Class, and the live raise
produces the new one. Identity fails, even though the error is
semantically correct.
This bites SafeFetch specifically because:
- SafeFetch is a *namespace* with 7 nested error classes — one reload
invalidates all of them.
- The spec contains 14+ `raise_error(SafeFetch::Foo)` assertions — many
chances to land in a reload window.
- SafeFetch is exercised by request-driven code (webhook delivery,
avatar fetch). Earlier specs in the suite warm up the reloader
machinery, which then fires during this spec.
Other custom-error specs don't visibly flake because their consumers
`rescue ConstName => e` (dynamic class lookup at raise time, walks the
ancestor chain via `Module#===` *at the moment of raise*, with no
captured snapshot), rather than RSpec's snapshot-then-compare pattern.
## What this PR does
Adds a small local helper to the spec that matches by class name string,
not by Class identity:
```ruby
def safe_fetch_error(name, message_pattern = nil)
satisfy("raise SafeFetch::#{name}#{" matching #{message_pattern.inspect}" if message_pattern}") do |error|
error.class.name == "SafeFetch::#{name}" &&
(message_pattern.nil? || message_pattern.match?(error.message))
end
end
```
Every `raise_error(described_class::FooError)` becomes
`raise_error(safe_fetch_error('FooError'))`. Class names are strings;
string equality survives any number of reloads. The semantic assertion
("this raised the right kind of error") is preserved.
This is the pattern CLAUDE.md already endorses for this codebase:
> Specs in parallel/reloading environments: prefer comparing
`error.class.name` over constant class equality when asserting raised
errors.
## Why this approach (even though it's suboptimal)
To be honest with reviewers: **this is a workaround, not a root-cause
fix.** Future spec authors who use `raise_error(SafeFetch::Foo)` in
other files will hit the same flake. The "real" fix removes the failure
mode at the source rather than dodging it per-spec.
Here's the menu of options we considered and the tradeoffs:
### Option A — Spec-side name matcher (this PR)
- **What:** the helper above.
- **Pro:** one-file change, zero blast radius beyond the spec.
- **Pro:** matches CLAUDE.md's documented stance.
- **Con:** every new spec touching reloadable error classes needs to
remember this pattern. It's a discipline tax, not a structural fix.
- **Verdict:** chosen for this PR.
### Option B — Pin SafeFetch outside Zeitwerk
```ruby
# config/application.rb
Rails.autoloaders.main.ignore(
Rails.root.join('lib/safe_fetch.rb'),
Rails.root.join('lib/safe_fetch'),
)
require Rails.root.join('lib/safe_fetch')
```
- **Pro:** real root-cause fix. `SafeFetch::*` constants become
process-lifetime stable. The spec helper becomes unnecessary, every
assertion in any spec works correctly.
- **Pro:** preserves the public API exactly.
- **Con:** loses hot-reload for SafeFetch in dev (need server restart to
see edits).
- **Con:** modifies `application.rb`, which has cross-team review
weight.
- **Verdict:** rejected for scope reasons in this PR; a sensible
follow-up.
### Option C — Move error classes to a non-reloadable location
Define top-level error classes in
`config/initializers/safe_fetch_errors.rb`. Initializers run once at
boot, constants are never reloaded.
- **Pro:** identity-stable errors, no spec helper needed.
- **Con:** API change — `SafeFetch::FileTooLargeError` becomes
`SafeFetchFileTooLargeError` (or similar). Every consumer's `rescue`
clause has to update.
- **Con:** namespace pollution at top-level.
- **Verdict:** rejected. The constraint of touching initializers is what
motivated the simpler PR.
### Option D — Vendor SafeFetch as a path gem
Move `lib/safe_fetch/` → `vendor/gems/safe_fetch/` with a gemspec.
Bundler `require`s gems once; Zeitwerk has zero involvement.
- **Pro:** structurally the most correct fix. Same identity stability as
Option B, no Zeitwerk plumbing required.
- **Pro:** zero API change for consumers.
- **Con:** larger refactor (6+ files moved, gemspec authored,
Gemfile/Gemfile.lock updated).
- **Con:** SafeFetch becomes harder to iterate on in dev (gem-style
edit-then-restart loop).
- **Verdict:** rejected for scope in this PR; the cleanest long-term
home for a security primitive.
### Option E — Drop custom errors, return a `Result`
Refactor SafeFetch to yield a result hash (`{ ok: true, ... }` / `{ ok:
false, kind: :unsafe_url, ... }`) instead of raising for anticipated
outcomes. Failure kinds become symbols, which are interned for the
process lifetime.
- **Pro:** eliminates the failure mode at its true root — there are no
custom exception classes to reload, anywhere.
- **Pro:** forces explicit, exhaustive handling at every call site — a
feature for a security primitive.
- **Pro:** spec assertions become data assertions (`expect(result).to
match(ok: false, kind: :too_large)`), which are robust against any
reload, any reordering.
- **Con:** paradigm shift away from the exception-driven style used
everywhere else in the codebase.
- **Con:** every consumer rewrites their `rescue` block into
pattern-matched handling.
- **Verdict:** rejected for scope in this PR; the architecturally
cleanest answer if we ever revisit SafeFetch's API.
## Why we're shipping A despite knowing B–E are better
This flake has been chewing CI time intermittently and previously took a
partial fix (#14139). We need it stable *now*. Options B–E are real
refactors with broader review surface (`application.rb`, consumer code,
or the lib's public contract). Option A:
- Costs nothing — one helper, mechanical replacements.
- Doesn't preclude any of B/C/D/E later. The helper goes away cleanly
once a root-cause fix lands.
- Aligns with the project's documented guidance for this scenario.
When SafeFetch next gets a substantive change, that's the right moment
to fold in B or D. Until then, the spec is stable and CI gets its time
back.
## What changed
- `spec/lib/safe_fetch_spec.rb`: added `safe_fetch_error(name,
message_pattern = nil)` helper; converted all 14
`raise_error(described_class::FooError[, /regex/])` assertions to
`raise_error(safe_fetch_error('FooError'[, /regex/]))`.
Stop the onboarding flow from writing the user's company website into
`accounts.domain`. That column is reserved for the inbound email domain
used to construct reply-to addresses (`reply+<uuid>@<domain>`), and
silently overloading it from onboarding was breaking email continuity
for accounts whose domain MX didn't point at Chatwoot's inbound —
customer replies were going to an unreachable address.
The website value now lives in `custom_attributes.website`, which is
what the rest of the app already treats as the "company website" field.
The bot metrics dashboard can show `handoff_rate + resolution_rate >
100%`. A single conversation can accumulate both
`conversation_bot_handoff` and `conversation_bot_resolved` events, and
the rate queries count them independently against a shared denominator.
## How it happens
```
Customer messages bot inbox
│
▼
┌──────────┐
│ pending │ (bot handling)
└────┬─────┘
│ bot can't help
▼
┌──────────┐
│ open │ (handed off → conversation_bot_handoff event created)
└────┬─────┘
│ agent clicks "Resolve" WITHOUT sending a message
▼
┌──────────┐
│ resolved │ conversation_resolved fires
└──────────┘
│
▼
create_bot_resolved_event guard checks:
✅ inbox.active_bot?
✅ no outgoing messages with sender_type: 'User' ← agent never messaged!
│
▼
conversation_bot_resolved event ALSO created ← BUG
│
▼
Same conversation counted in BOTH rates → sum exceeds 100%
```
## Why fix at the read path, not the write path
An earlier attempt added guards in the listener to make the two events
mutually exclusive per conversation — deleting `bot_resolved` when a
handoff fires, suppressing resolutions when a handoff exists. This was
rejected because conversations can be reopened across multiple cycles
(bot resolves on day 1, customer returns on day 5, bot hands off).
Deleting the day-1 resolution corrupts historical reports, and the async
event dispatcher makes listener-level guards vulnerable to race
conditions.
## What this PR does
Within a reporting window, if a conversation has both events, **handoff
wins** — the conversation is excluded from the resolution count. This is
applied via SQL subquery across all three read paths:
```
┌─────────────────────────┐
│ Reporting Events DB │
│ │
│ conv_bot_handoff: [A,B] │
│ conv_bot_resolved: [A,C]│
└────────┬────────────────┘
│
┌──────────────┼──────────────┐
▼ ▼ ▼
BotMetricsBuilder ReportHelper CountReportBuilder
(rate cards) (bot_summary) (timeseries charts)
│ │ │
▼ ▼ ▼
resolutions: resolutions: resolutions:
[A,C] minus [A,B] same logic same logic
= [C] only = [C] only = [C] only
Result: Conversation A → handoff only
Conversation B → handoff only
Conversation C → resolution only
```
For wide date ranges spanning multiple lifecycles, a conversation
bot-resolved in one cycle and handed off in a later cycle will only show
as a handoff. This is an acceptable tradeoff — the alternative (>100%
rates) is clearly worse, and narrow ranges handle this correctly since
the events fall into different windows. No reporting events are
modified, so historical data stays intact.
## Diagnostic tool
`rake bot_metrics:diagnose` — read-only task that prompts for account ID
and date range, shows a before/after rate comparison without modifying
data.
---------
Co-authored-by: aakashb95 <aakashbakhle@gmail.com>
Co-authored-by: Aakash Bakhle <48802744+aakashb95@users.noreply.github.com>
When a new account finishes onboarding we want to land them on a
dashboard with a working web widget already configured, branded, named,
and assigned to them, instead of an empty inbox list. This PR adds the
services that produce that widget. **No user-visible change yet:** the
services are dormant until the trigger and background job are wired up
in the follow-up PR.
## Context
Milestone 1 added `Account::BrandingEnrichmentJob`, which calls
context.dev during signup and stores brand data on
`account.custom_attributes['brand_info']`, plus the new onboarding form
that captures `domain`, `name`, `industry`, etc. Milestone 2 starts
using that data, and the first thing we want is a web widget
materialized automatically. Splitting the service layer from the
orchestration plumbing (Redis key, `onboarding_step` extension,
controller wiring, ActionCable) keeps this diff focused and lets the
LLM/widget logic merge independently.
## How to test
Run against an existing account that already has `brand_info` populated.
```ruby
account = Account.find(<account_id>)
user = account.administrators.first
inbox = WidgetCreationService.new(account, user).perform
inbox.channel.widget_color # color from brand_info, or '#1f93ff'
inbox.channel.welcome_title # brand_info[:title], or account.name
inbox.channel.welcome_tagline # LLM tagline (Enterprise + system key set),
# else brand_info[:slogan]/[:description]/nil
inbox.inbox_members.pluck(:user_id)
```
Toggle `InstallationConfig['CAPTAIN_OPEN_AI_API_KEY']` to flip between
LLM and brand-text tagline paths. To verify failure isolation, raise
inside `Captain::Llm::WidgetTaglineService#perform` and confirm widget
creation still succeeds with the fallback tagline.
The SafeFetch spec suite was failing in CI with `NameError:
uninitialized constant SafeFetch::Fetcher` across every example that
exercised `SafeFetch.fetch`. From a product perspective, this made the
external-file fetch path look unreliable even though the failure
happened before any network validation, SSRF protection, content-type
checks, or tempfile handling could run.
The symptom pointed to a load-order issue rather than an actual fetch
behavior regression. `SafeFetch.fetch` referenced `Fetcher` from the
top-level module, but that nested class was not guaranteed to be loaded
in every test execution path before the method was invoked.
This change keeps the existing SafeFetch split between the public API
and the implementation classes, but makes the public entry point
responsible for loading the implementation it needs before use. That is
intentionally smaller than folding all of the fetcher logic into
`lib/safe_fetch.rb`; the separate files still keep the request option
parsing and streaming implementation readable, while the public API no
longer depends on Rails or the test runner having loaded nested
constants in a particular order.
The file also now uses a single `SafeFetch` module declaration. That
removes the awkward reopen pattern and makes the dependency boundary
easier to see: constants and errors are defined first, then the public
`fetch` method loads and delegates to the implementation classes.
The "conversation continuity via email" toggle was visible to all
accounts regardless of whether they had `inbound_emails` enabled.
Without inbound email infrastructure, replies to those follow-up emails
land in the agent's personal inbox instead of routing back into
Chatwoot. The feature appears to work but silently breaks the reply
path.
The toggle is now gated on the `inbound_emails` feature flag. On
self-hosted without the feature, the toggle is hidden entirely. On
cloud, it remains visible but disabled with upgrade messaging.
On the backend, `inbound_emails` is added to the manually managed
features list in `InternalAttributesService` so that Stripe webhook plan
syncs don't override it when support enables it for an account.
---------
Co-authored-by: Sivin Varghese <64252451+iamsivin@users.noreply.github.com>
## PR2: Report builder refactor — DataSource abstraction
The existing report builders (timeseries + summary) had their SQL
queries inlined — each builder constructed its own scopes, groupings,
and aggregations directly. This made it hard to swap the underlying data
source without duplicating builder logic.
This PR extracts all raw-event querying into a `Reports::RawDataSource`
behind a `Reports::DataSource` factory. Builders now call
`data_source.timeseries`, `.aggregate`, or `.summary` instead of
constructing queries themselves. Behavior is identical —
`DataSource.for(...)` returns `RawDataSource` in all cases today.
The timeseries path had two separate builders (`CountReportBuilder`,
`AverageReportBuilder`) that were selected via a metric-name case
statement in `Conversations::BaseReportBuilder`. These are replaced by a
single `ReportBuilder` that delegates to the data source. The metric
type (count vs average) is now decided inside the data source, not the
builder.
Summary builders similarly moved their inline SQL into
`RawDataSource#summary`, which returns a unified hash keyed by dimension
ID.
the rollup read path.
## Flow
### Before
```
ReportsController ──▶ case metric ──▶ AverageReportBuilder ──▶ inline SQL ──▶ DB
└──▶ CountReportBuilder ──▶ inline SQL ──▶ DB
SummaryController ──▶ AgentSummaryBuilder ──▶ inline SQL ──▶ DB
└──▶ InboxSummaryBuilder ──▶ inline SQL ──▶ DB
└──▶ TeamSummaryBuilder ──▶ inline SQL ──▶ DB
```
### After
```
ReportsController ──▶ ReportBuilder ──┐
├──▶ DataSource.for ──▶ RawDataSource ──▶ DB
SummaryController ──▶ SummaryBuilder ──┘
```
### Expected (after rollup read path)
```
ReportsController ──▶ ReportBuilder ──┐
├──▶ DataSource.for ──▶ RawDataSource ──▶ reporting_events
SummaryController ──▶ SummaryBuilder ──┘ └──▶ RollupDataSource ──▶ reporting_events_rollups
```
### What changed
- `Reports::DataSource` factory + `Reports::RawDataSource`
- `TimezoneHelper#timezone_name_from_params` — prefers IANA name, falls
back to offset
- Unified `Timeseries::ReportBuilder` replaces `CountReportBuilder` +
`AverageReportBuilder`
- Summary builders delegate to `DataSource` instead of querying directly
### How to test
This is a pure refactor — all existing report pages (Overview, Agent,
Inbox, Label, Team) should produce identical numbers. No feature flag or
new config needed.
---------
Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
Co-authored-by: Tanmay Deep Sharma <tanmaydeepsharma21@gmail.com>
Co-authored-by: Tanmay Deep Sharma <32020192+tds-1@users.noreply.github.com>
## Account branding enrichment during signup
This PR does the following
### Replace Firecrawl with Context.dev
Switches the enterprise brand lookup from Firecrawl to Context.dev for
better data quality, built-in caching, and automatic filtering of
free/disposable email providers. The service interface changes from URL
to email input to match Context.dev's email endpoint. OSS still falls
back to basic HTML scraping with a normalized output shape across both
paths.
The enterprise path intentionally does not fall back to HTML scraping on
failure — speed matters more than completeness. We want the user on the
editable onboarding form fast, and a slow fallback scrape is worse than
letting them fill it in.
Requires `CONTEXT_DEV_API_KEY` in Super Admin → App Config. Without it,
falls back to OSS HTML scraping.
### Add job to enrich account details
After account creation, `Account::BrandingEnrichmentJob` looks up the
signup email and pre-fills the account name, colors, logos, social
links, and industry into `custom_attributes['brand_info']`.
The job signals completion via a short-lived Redis key (30s TTL) + an
ActionCable broadcast (`account.enrichment_completed`). The Redis key
lets the frontend distinguish "still running" from "finished with no
results."
Previously, signing up gave immediate access to the app. Now,
unconfirmed users are redirected to a verification page where they can
resend the confirmation email.
- After signup, the user is routed to `/auth/verify-email` instead of
the dashboard
- After login, unconfirmed users are redirected to the verification page
- The dashboard route guard catches unconfirmed users and redirects them
- `active_for_authentication?` is removed from the sessions controller
so unconfirmed users can authenticate — the frontend gates access
instead
- If the user visits the verification page after already confirming,
they're automatically redirected to the dashboard
- No session is issued until the user is verified
<details><summary>Demo</summary>
<p>
#### Fresh Signup
https://github.com/user-attachments/assets/abb735e5-7c8e-44a2-801c-96d9e4823e51
#### Google Fresh Signup
https://github.com/user-attachments/assets/ab9e389a-a604-4a9d-b492-219e6d94ee3f
#### Create new account from Dashboard
https://github.com/user-attachments/assets/c456690d-1946-4e0b-834b-ad8efcea8369
</p>
</details>
---------
Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
Account webhooks sign outgoing payloads with HMAC-SHA256, but agent bot
and API inbox webhooks were delivered unsigned. This PR adds the same
signing to both.
Each model gets a dedicated `secret` column rather than reusing the
agent bot's `access_token` (for API auth back into Chatwoot) or the API
inbox's `hmac_token` (for inbound contact identity verification). These
serve different trust boundaries and shouldn't be coupled — rotating a
signing secret shouldn't invalidate API access or contact verification.
The existing `Webhooks::Trigger` already signs when a secret is present,
so the backend change is just passing `secret:` through to the jobs.
Shared token logic is extracted into a `WebhookSecretable` concern
included by `Webhook`, `AgentBot`, and `Channel::Api`. The frontend
reuses the existing `AccessToken` component for secret display. Secrets
are admin-only and excluded from enterprise audit logs.
### How to test
Point an agent bot or API inbox webhook URL at a request inspector. Send
a message and verify `X-Chatwoot-Signature` and `X-Chatwoot-Timestamp`
headers are present. Reset the secret from settings and confirm
subsequent deliveries use the new value.
---------
Co-authored-by: Sojan Jose <sojan@pepalo.com>
When a user signs up with an email they don't own and sets a password,
that password remains valid even after the real owner later signs in via
OAuth. This means the original registrant — who never proved ownership
of the email — retains working credentials on the account. This change
closes that gap by rotating the password to a random value whenever an
unconfirmed user completes an OAuth sign-in.
The check (`oauth_user_needs_password_reset?`) is evaluated before
`skip_confirmation!` runs, since confirmation would flip `confirmed_at`
and mask the condition. If the user was unconfirmed, the stored password
is replaced with a secure random string that satisfies the password
policy. This applies to both the web and mobile OAuth callback paths, as
well as the sign-up path where the password is rotated before the reset
token is generated.
Users who lose access to password-based login as a side effect can
recover through the standard "Forgot password" flow at any time. Since
they've already proven email ownership via OAuth, this is a low-friction
recovery path
Adds `WebsiteBrandingService` (OSS) with an Enterprise override using
Firecrawl v2 to extract branding and business data from a URL for
onboarding auto-fill.
OSS version uses HTTParty + Nokogiri to extract:
- Business name (og:site_name or title)
- Language (html lang)
- Favicon
- Social links from `<a>` tags
Enterprise version makes a single Firecrawl call to fetch:
- Structured JSON (name, language, industry via LLM)
- Branding (favicon, primary color)
- Page links
Falls back to OSS if Firecrawl is unavailable or fails.
Social handles (WhatsApp, Facebook, Instagram, Telegram, TikTok, LINE)
are parsed deterministically via a shared `SocialLinkParser`.
> We use links for socials, since the LLM extraction was unreliable,
mostly returned empty, and hallucinated in some rare scenarios
## How to test
```ruby
# OSS (no Firecrawl key needed)
WebsiteBrandingService.new('chatwoot.com').perform
# Enterprise (requires CAPTAIN_FIRECRAWL_API_KEY)
WebsiteBrandingService.new('notion.so').perform
WebsiteBrandingService.new('postman.com').perform
```
Verify the returned hash includes business_name, language,
industry_category, social_handles, and branding with
favicon/primary_color.
<img width="908" height="393" alt="image"
src="https://github.com/user-attachments/assets/e3696887-d366-485a-89a0-8e1a9698a788"
/>
When syncing contacts to LeadSquared, the `Lead.CreateOrUpdate` API
defaults to searching by email. If a contact has no email (or a
different email) but a phone number matching an existing lead, the API
fails with `MXDuplicateEntryException` instead of finding and updating
the existing lead. This accounted for ~69% of all LeadSquared
integration errors, and cascaded into "Lead not found" failures when
posting transcript and conversation activities (~14% of errors).
## What changed
- `LeadClient#create_or_update_lead` now catches
`MXDuplicateEntryException` and retries the request once with
`SearchBy=Phone` appended to the body, telling the API to match on phone
number instead
- Once the retry succeeds, the returned lead ID is stored on the contact
(existing behavior), so all future events use the direct `update_lead`
path and never hit the duplicate error again
## How to reproduce
1. Create a lead in LeadSquared with phone number `+91-75076767676` and
email `a@example.com`
2. In Chatwoot, create a contact with the same phone number but a
different email (or no email)
3. Trigger a contact sync (via conversation creation or contact update)
4. Before fix: `MXDuplicateEntryException` error in logs, contact fails
to sync
5. After fix: retry with `SearchBy=Phone` finds and updates the existing
lead, stores the lead ID on the contact
## PR#1: Reporting events rollup — model and write path
Reporting queries currently hit the `reporting_events` table directly.
This works, but the table grows linearly with event volume, and
aggregation queries (counts, averages over date ranges) get
progressively slower as accounts age.
This PR introduces a pre-aggregated `reporting_events_rollups` table
that stores daily per-metric, per-dimension (account/agent/inbox)
totals. The write path is intentionally decoupled from the read path —
rollup rows are written inline from the event listener via upsert, and a
backfill service exists to rebuild historical data from raw events.
Nothing reads from this table yet.
The write path activates when an account has a `reporting_timezone` set
(new account setting). The `reporting_events_rollup` feature flag
controls only the future read path, not writes — so rollup data
accumulates silently once timezone is configured. A `MetricRegistry`
maps raw event names to rollup column semantics in one place, keeping
the write and (future) read paths aligned.
### What changed
- Migration for `reporting_events_rollups` with a unique composite index
for upsert
- `ReportingEventsRollup` model
- `reporting_timezone` account setting with IANA timezone validation
- `MetricRegistry` — single source of truth for event-to-metric mappings
- `RollupService` — real-time upsert from event listener
- `BackfillService` — rebuilds rollups for a given account + date from
raw events
- Rake tasks for interactive backfill and timezone setup
- `reporting_events_rollup` feature flag (disabled by default)
### How to test
1. Set a `reporting_timezone` on an account
(`Account.first.update!(reporting_timezone: 'Asia/Kolkata')`)
2. Resolve a conversation or trigger a first response
3. Check `ReportingEventsRollup.where(account_id: ...)` — rows should
appear
4. Run backfill: `bundle exec rake reporting_events_rollup:backfill` and
verify historical data populates
---------
Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
Devise 4.9.x has a race condition in the reconfirmable flow where
concurrent email change requests can desynchronize the confirmation
token from `unconfirmed_email`, letting an attacker confirm an email
they don't own. We use `:confirmable` with `reconfirmable = true`, so
we're directly exposed.
The upstream fix is in Devise 5.0.3, but we can't upgrade —
`devise-two-factor` only supports Devise 5 from v6.4.0, which also
raised its Rails minimum to 7.2+. No released version supports both
Devise 5 and Rails 7.1.
This PR ports the Devise 5.0.3 fix locally by overriding
`postpone_email_change_until_confirmation_and_regenerate_confirmation_token`
on the User model to persist the record before regenerating the token.
This is a stopgap — remove it once the dependency chain allows upgrading
to Devise 5.
### How to test
Sign in as a confirmed user and change your email. The app should send a
confirmation to the new address while keeping the current email
unchanged until confirmed.
SAML sign-in now only links an existing user when that user already
belongs to the account that initiated SSO. New users can still be
created for SAML-enabled accounts, and invited members can continue to
sign in through their IdP, but SAML will no longer auto-attach an
unrelated existing user record during login.
**What changed**
- Added an account-membership check before SAML reuses an existing user
by email.
- Kept first-time SAML user creation unchanged for valid new users.
- Added builder and request specs covering the allowed and rejected
login paths.
This PR generally upgrades the available packages clearing existing
transient package vulnerabilities
### minimatch
This is mostly a dev-dependency, it was in the main dependency for
`@formkit/vue`, this PR clears it.
It still remains a dependency for tailwind and tailwind typograpghy, but
it's tolerable
_Fixes: CW-6594 CW-6592 CW-6598_
### Axios
Upgraded the dependency directly. Fixed it
_Fixes: CW-6591_
### dompurify
Upgraded the dependency, it's in the safe range now
_Fixes: CW-6611 CW-6610_
### ajv
Dev dependency, can be safely ignored
_Fixes: CW-6606 CW-6604_
### @tootallnate/once
Dev dependency, comes from Histoire, can be safely ignored
_Fixes: CW-6603_
Adds Skooma-based OpenAPI validation so SDK-facing request specs can
assert that documented request and response contracts match real Rails
behavior. This also upgrades the spec to OpenAPI 3.1 and fixes contract
drift uncovered while validating core application and platform
resources.
Closes
None
Why
We want CI to catch OpenAPI drift before it reaches SDK consumers. While
wiring validation in, this PR surfaced several mismatches between the
documented contract and what the Rails endpoints actually accept or
return.
What this change does
- Adds Skooma-backed OpenAPI validation to the request spec flow and a
dedicated OpenAPI validation spec.
- Migrates nullable schema definitions to OpenAPI 3.1-compatible unions.
- Updates core SDK-facing schemas and payloads across accounts,
contacts, conversations, inboxes, messages, teams, reporting events, and
platform account resources.
- Documents concrete runtime cases that were previously missing or
inaccurate, including nested `profile` update payloads, multipart avatar
uploads, required profile update bodies, nullable inbox feature flags,
and message sender types that include both `Captain::Assistant` and
senderless activity-style messages.
- Regenerates the committed Swagger JSON and tag-group artifacts used by
CI sync checks.
Validation
- `bundle exec rake swagger:build`
- `bundle exec rspec spec/swagger/openapi_spec.rb`
---------
Co-authored-by: Sojan Jose <sojan@pepalo.com>
This makes account signup enforcement consistent when signup is disabled
at the installation level. Email signup and Google signup now stay
blocked regardless of whether the config value is stored as a string or
a boolean.
This effectively covers the config-loader path, where `YAML.safe_load`
reads `value: false` from `installation_config.yml` as a native boolean
and persists it that way.
- Normalized the account signup check so disabled signup is handled
consistently across config value types.
- Reused the same check across API signup and Google signup entry
points.
- Added regression coverage for the disabled-signup cases in the
existing controller specs.
---------
Co-authored-by: Vishnu Narayanan <iamwishnu@gmail.com>
- Extracted 6 custom attribute methods (`custom_attribute_query`,
`attribute_model`, `attribute_data_type`, `build_custom_attr_query`,
`custom_attribute`, `not_in_custom_attr_query`) into a new
`Filters::CustomAttributeFilterHelper` module.
- Added an inline `rubocop:disable` for the intentional
`Lint/ShadowedException` in `coerce_lt_gt_value` — `Date::Error` is a
subclass of `ArgumentError`, but both are listed explicitly for clarity.
## Why `app/services/filters/`
The existing `Filters::FilterHelper` lives in `app/helpers/filters/`,
but that location triggers `Rails/HelperInstanceVariable` for any module
that uses instance variables. The extracted methods share state with
`FilterService` via instance variables (`@attribute_key`, `@account`,
`@custom_attribute`, etc.), so placing them in `app/helpers/` would
require a cop disable.
`app/services/filters/` is a better fit because:
- The module is a service mixin, not a view helper — it's only included
by `FilterService` and its subclasses (`Conversations::FilterService`,
`Contacts::FilterService`, `AutomationRules::ConditionsFilterService`).
- It sits alongside the services that use it.
- No cop disables needed.
---------
Co-authored-by: Vishnu Narayanan <iamwishnu@gmail.com>
Some customers using WhatsApp inboxes with account-level webhooks were
reporting receiving duplicate `message_created` webhook deliveries for
every incoming message. Upon inspection, here's what we found
- Both payloads are identical.
- No errors appear in the application logs
- Webhook URL is only configured in one place.
This meant, the system was sending the webhooks twice. For some context,
there's a know related issue... Meta's WhatsApp Business API can deliver
the same webhook notification multiple times for a single message. The
codebase already acknowledges this — there's a comment in
`IncomingMessageBaseService#process_messages` noting that "multiple
webhook events can be received against the same message due to
misconfigurations in the Meta business manager account." A deduplication
guard exists, but it doesn't actually work under concurrency.
### Rationale
The existing dedup was a three-step sequence: check Redis (`GET`), check
the database, then set a Redis flag (`SETEX`). Two Sidekiq workers
processing duplicate Meta webhooks simultaneously would both complete
the `GET` before either executed the `SETEX`, so both would proceed to
create a message. The `source_id` column has a non-unique index, so the
database wouldn't catch the duplicate either. Each message then
independently fires `after_create_commit`, dispatching two
`message_created` webhook events to the customer.
```
Worker A Worker B
│ │
▼ ▼
Redis GET key ──► nil Redis GET key ──► nil
│ │
│ ◄── both pass guard ──► │
│ │
▼ ▼
Redis SETEX key Redis SETEX key
│ │
▼ ▼
BEGIN transaction BEGIN transaction
INSERT message INSERT message
DELETE Redis key ◄─┐ │
COMMIT │ DELETE Redis key
│ COMMIT
│ │
└── key gone before ───┘
B's commit lands
▼ ▼
after_create_commit after_create_commit
dispatch MESSAGE_CREATED dispatch MESSAGE_CREATED
│ │
▼ ▼
WebhookJob ──► n8n WebhookJob ──► n8n
(duplicate!)
```
There was a second, subtler problem visible in the diagram: the Redis
key was cleared *inside* the database transaction, before the
transaction committed. This opened a window where neither the Redis
check nor the database check would see the in-flight message.
The fix collapses the check-and-set into a single `SET NX EX` call,
which is atomic in Redis. The key is no longer eagerly cleared — it
expires naturally after 24 hours. The database lookup
(`find_message_by_source_id`) remains as a fallback for messages that
were created before the lock expired.
```
Worker A Worker B
│ │
▼ ▼
Redis SET NX ──► OK Redis SET NX ──► nil
│ │
▼ ▼
proceeds to create returns early
message normally (lock already held)
```
### Implementation Notes
The lock logic is extracted into `Whatsapp::MessageDedupLock`, a small
class that wraps a single `Redis SET NX EX` call. This makes the
concurrency guarantee testable in isolation — the spec uses a
`CyclicBarrier` to race two threads against the same key and asserts
exactly one wins, without needing database writes,
`use_transactional_tests = false`, or monkey-patching.
Because the Redis lock now persists (instead of being cleared
mid-transaction), existing WhatsApp specs needed an `after` hook to
clean up `MESSAGE_SOURCE_KEY::*` keys between examples. Transactional
fixtures only roll back the database, not Redis.
Creating a Linear issue from Chatwoot fails with a GraphQL parse error
when the title, description, or search term contains double quotes. For
example, a description like `the sender is "Bot"` produces this broken
query:
```graphql
issueCreate(input: { description: "the sender is "Bot"" })
```
Linear's API rejects this with `Syntax Error: Expected ":", found
String`. This affects issue creation, issue linking, and issue search —
any flow where user-provided text is interpolated into a GraphQL query.
The `graphql_value` helper was only escaping newlines (`\n`) but not
quotes, backslashes, or other characters that are meaningful inside a
GraphQL string literal. On top of that, `issue_link` and `search_issue`
bypassed `graphql_value` entirely, using raw string interpolation
instead.
The fix replaces the manual `gsub` escaping with Ruby's `to_json`, which
produces a properly escaped, double-quoted string that handles all
special characters. This is a minimal, well-understood substitution —
`to_json` on a Ruby string returns a valid JSON string literal, which is
also a valid GraphQL string literal since GraphQL uses the same escaping
rules. The `issue_link` mutation and `search_issue` query are updated to
route their parameters through `graphql_value` instead of raw
interpolation.
The `team_entities_query` and `linked_issues` methods in `queries.rb`
also use raw interpolation, but their inputs are system-generated IDs
and URLs rather than user-provided text, so they're left as-is to keep
this change focused.
For large accounts, summary report queries can take several seconds to
complete, often times hitting the 15-second production request timeout.
The existing implementation silently swallows these failures and
provides no feedback during loading. Users see stale data with no
indication that a fetch is in progress, and if they interact with
filters while a request is in flight, they trigger race conditions that
can result in mismatched data being displayed.
This is a UX-level fix for what is fundamentally a performance problem.
While the underlying query performance is addressed separately, users
need proper feedback either way
## Approach
The PR adds three things:
1. A loading overlay on the table, to provide feedback on loading state
2. Disabled filter inputs during loading so that the user does not
request new information that can cause race conditions in updating the
store
3. Silent retry before showing an error.
The retry exists because these queries often succeed on the second
attempt—likely due to database query caching. Rather than immediately
showing an error and forcing the user to manually retry, we do it
automatically. If the second attempt also fails, we show a toast so the
user knows something went wrong.
The store previously caught and discarded errors entirely. It now
rethrows them after resetting the loading flag, allowing components to
handle failures as they see fit.
### Previews
#### Double Retry and Error
https://github.com/user-attachments/assets/c189b173-8017-44b7-9493-417d65582c95
#### Loading State
https://github.com/user-attachments/assets/9f899c20-fbad-469b-93cc-f0d05d0853b0
---------
Co-authored-by: iamsivin <iamsivin@gmail.com>
We've been seeing orphan conversations (conversations whose contact has
been deleted) on high-frequency accounts. These orphans cause 500 errors
when the API attempts to render conversation data, since the jbuilder
partials expect a valid contact association.
The `ProcessStaleContactsJob` triggers `RemoveStaleContactsService`,
which uses `delete_all` to remove contacts. While the query only selects
contacts without conversations, `delete_all` bypasses ActiveRecord
callbacks and does not re-verify at deletion time. We suspect this
creates a window where a new conversation can be created for a contact
between query evaluation and the actual delete, leaving the conversation
orphaned.
**This is unverified — disabling the job is the experiment to confirm or
rule out this theory.**
This PR disables the job for a monitoring period. If orphan
conversations stop appearing, we'll have confirmation and can work on a
proper fix for the service. If they continue, we'll investigate other
deletion paths.
Stale contacts will accumulate while the job is disabled, but this is a
controlled tradeoff — they are inert records with no user-facing impact,
and can be cleaned up in bulk once we re-enable or fix the job.
When AgentBot responds to customer messages, the `waiting_since`
timestamp is not reset, causing inflated reply time metrics when a human
agent eventually responds. This results in inaccurate reporting that
incorrectly includes periods when customers were satisfied with bot
responses.
### Timeline from Production Data
```
Dec 12, 16:20:14 - Customer sends message (ID: 368451924)
↓ waiting_since = Dec 12, 16:20:14
Dec 12, 16:20:17 - AgentBot replies (ID: 368451960)
↓ waiting_since STILL = Dec 12, 16:20:14 ❌
↓ (Bot response doesn't clear it)
14-day gap - Customer satisfied, no messages
↓ waiting_since STILL = Dec 12, 16:20:14 ❌
Dec 26, 22:25:45 - Customer sends new message (ID: 383522275)
↓ waiting_since STILL = Dec 12, 16:20:14 ❌
↓ (New message doesn't reset it)
Dec 26-27 - More AgentBot interactions
↓ waiting_since STILL = Dec 12, 16:20:14 ❌
Dec 27, 07:36:53 - Human agent finally replies (ID: 383799517)
↓ Reply time calculated: 1,268,404 seconds
↓ = 14.7 DAYS ❌
```
## Root Cause
The core issues is in `app/models/message.rb`, where **AgentBot messages
does not clear `waiting_since`** - The `human_response?` method only
returns true for `User` senders, so bot replies never trigger the
clearing logic. This means once `waiting_since` is set, it stays set
even when customers send new messages after receiving bot responses.
The solution is to simply reset `waiting_since` **after a bot has
responded**. This ensures reply time metrics reflect actual human agent
response times, not bot-handled periods.
### What triggers the rest
This is an intentional "gotcha", that only `AgentBot` and
`Captain::Assistant` messages trigger the waiting time reset. Automation
and campaign messages maintain current behavior (no reset). This is
because interactive bot assistants provide conversational help that
might satisfy customers. Automation and campaigns are one-way
communications and shouldn't affect waiting time calculations.
## Related Work
Extends PR #11787 which fixed `waiting_since` clearing on conversation
resolution. This PR addresses the bot interaction scenario which was not
covered by that fix.
Scripts to clean data:
https://gist.github.com/scmmishra/bd133208e219d0ab52fbfdf03036c48a
