## Description
Reduces sustained load during large Intercom imports by limiting
conversation list pages to 10 while retaining 50-contact pages. Cursor
and provider total-count behavior remain unchanged.
The one-minute progress heartbeat now lives in parent PR #15050 because
stalled-import detection must be safe when that PR is deployed
independently. This child PR therefore contains only the smaller-page
delta.
This is Phase 1, Task 2 of the [Intercom import optimization plan
(CW-7615)](https://linear.app/chatwoot/issue/CW-7615/optimize-intercom-import-reliability-and-bulk-message-ingestion).
This PR is stacked on #15050 and should be reviewed as the two-file
delta from `codex/cw-7519-intercom-stalled-retry-15m`.
## Closes
-
[CW-7615](https://linear.app/chatwoot/issue/CW-7615/optimize-intercom-import-reliability-and-bulk-message-ingestion)
## Type of change
- [x] Bug fix (non-breaking change which fixes an issue)
## How to test
1. Start an Intercom import containing contacts and conversations across
multiple source pages.
2. Confirm contact list requests retain a page size of 50.
3. Confirm conversation list requests use a page size of 10.
4. Confirm page cursors and displayed provider totals continue advancing
normally.
5. Confirm the parent PR heartbeat keeps long conversations fresh while
these shorter pages are processed.
## Checklist
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have added tests that prove the change is effective
- [x] New and existing focused tests pass locally with my changes
## Description
Prepares each Intercom conversation's source message and parts as one
deterministic batch before persistence. The builder preserves provider
order, prefetches mappings and live messages once, and classifies every
entry as current-run, previous-run, stale-mapping repair,
existing-message repair, or new.
The importer consumes those prepared entries through the existing
individual transaction and retry path, so activity conversion, private
notes, sender attribution, metadata, timestamps, skip logs, idempotency,
indexing, and heartbeat behavior remain unchanged. This PR intentionally
does not bulk insert messages; batching writes is Phase 2 Task 2.
This is Phase 2, Task 1 (Task 4 overall) of
[CW-7615](https://linear.app/chatwoot/issue/CW-7615/optimize-intercom-import-reliability-and-bulk-message-ingestion).
This PR is stacked on #15052 and should be reviewed as the four-file
delta from `codex/cw-7615-intercom-query-timeout-retries`.
## Closes
- Tracking plan:
[CW-7615](https://linear.app/chatwoot/issue/CW-7615/optimize-intercom-import-reliability-and-bulk-message-ingestion)
## Type of change
- [x] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality not to work as expected)
- [ ] This change requires a documentation update
## How to test
1. Import a conversation containing a source message, comments, notes,
activities, and entries with equal timestamps.
2. Retry the same run and verify messages and counters remain
idempotent.
3. Retry after a previous run and verify existing mappings are reported
as skips without changing their owner.
4. Delete a mapped Message, rerun, and verify the message and mapping
are repaired.
5. Delete only a mapping, rerun, and verify the existing Message is
reused without duplication.
6. Rerun a previously skipped part that now classifies as an activity
and verify it is repaired.
7. Import an empty conversation and verify no message or mapping
prefetch is performed.
## Checklist
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged and published in downstream
modules
## Description
Reduces database pressure during large Intercom imports by deferring
same-run contact and message statistic recounts until the page boundary,
immediately before the cursor advances. Repeated mappings within a page
now mark their statistic group dirty instead of running aggregate counts
and persisting statistics for every record.
Contact and message database attempts now retry once when PostgreSQL
cancels a query. The failed transaction is allowed to roll back before
the importer waits for a randomized 200-500 ms delay and retries. A
recovered timeout creates no error; a second timeout follows the
existing item/message failure path exactly once. Provider, validation,
and other database errors are not retried.
This is Phase 1, Task 3 of the [Intercom import optimization plan
(CW-7615)](https://linear.app/chatwoot/issue/CW-7615/optimize-intercom-import-reliability-and-bulk-message-ingestion).
This PR is stacked on #15051 and should be reviewed as the two-file
delta from `codex/cw-7519-intercom-shorter-jobs-heartbeats`.
## Closes
- Tracking plan:
[CW-7615](https://linear.app/chatwoot/issue/CW-7615/optimize-intercom-import-reliability-and-bulk-message-ingestion)
## Type of change
- [x] Bug fix (non-breaking change which fixes an issue)
## How to test
1. Retry a page whose contacts or messages already have same-run
mappings and confirm imported/skipped statistics are reconciled before
the page cursor advances.
2. Raise one `ActiveRecord::QueryCanceled` while persisting a contact
mapping and confirm the transaction rolls back, retries, and creates one
contact and mapping without an error.
3. Repeat for a message mapping and confirm one message and mapping are
created without an error.
4. Raise the timeout on both attempts and confirm one final item/message
error is recorded.
5. Raise an unrelated database error and confirm it is not retried.
## Checklist
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have added tests that prove the change is effective
- [x] New and existing focused tests pass locally with my changes
This preserves per-message failure isolation when an Intercom bulk
message chunk cannot be committed. Query timeouts are retried once after
the failed transaction rolls back, while other database failures move
directly to an entry-by-entry fallback.
The fallback reacquires the import lock and refreshes each entry before
writing, so a competing worker cannot create duplicate messages. Each
individual write uses a savepoint, and search indexing happens only
after the import lock transaction commits.
This is Phase 2, Task 6 of CW-7615 and is stacked on #15111. The review
delta is one commit across the importer and its focused spec.
## Closes
-
[CW-7615](https://linear.app/chatwoot/issue/CW-7615/optimize-intercom-import-reliability-and-bulk-message-ingestion)
## How to test
1. Run an Intercom import where a bulk mapping write times out once and
confirm the chunk succeeds on retry.
2. Force a persistent timeout or database write failure and confirm
valid messages are imported through the fallback.
3. Force one fallback record to fail and confirm only that message is
recorded as an import error.
4. Simulate another worker repairing a message before fallback and
confirm no duplicate is created.
WhatsApp automations now fail locally when they attempt to send a
free-form message after the 24-hour customer service window has closed.
This avoids sending an invalid template request to Meta and gives users
a clear, actionable error instead of “Template not found or invalid
template name.”
Template messages continue to be sent whenever template parameters are
present. Free-form messages continue to be sent normally while the
conversation is replyable.
Fixes
https://linear.app/chatwoot/issue/PLA-183/prevent-whatsapp-automations-outside-the-24-hour-window-from-producing
### How to reproduce
1. Create a WhatsApp automation that sends a message without template
parameters.
2. Trigger it on a conversation whose 24-hour customer service window is
closed.
3. Observe that the message previously reached the template send path
and failed with a misleading provider error.
### How to test
1. Trigger an automation with template parameters and confirm it sends
as a template message.
2. Trigger an automation without template parameters inside the 24-hour
window and confirm it sends as a free-form message.
3. Trigger an automation without template parameters outside the 24-hour
window and confirm it fails locally with a clear error and makes no
request to Meta.
### Things to know
This changes only the invalid closed-window, no-template path. Existing
template and in-window message behavior remains unchanged.
---------
Co-authored-by: Muhsin <12408980+muhsin-k@users.noreply.github.com>
Fixes
https://linear.app/chatwoot/issue/PLA-99/whatsapp-messages-dropped-for-brazilargentina-numbers-due-to-phone
Fixes https://github.com/chatwoot/chatwoot/issues/14492
Meta's WhatsApp Cloud API includes `display_phone_number` in webhook
payloads, but its format can differ from the number stored in Chatwoot's
channel record.
In Brazil, Meta omits the mobile 9 prefix. For example, it sends
55419XXXXXXX (12 digits) instead of 554199XXXXXXX (13 digits). In
Argentina, Meta adds an extra 9 after the country code. For example, it
sends 549XXXXXXXXXX instead of 54XXXXXXXXXX.
The whatsapp event job uses `display_phone_number` for an exact-match
channel lookup. When the formats do not match, the lookup returns nil
and the incoming message is silently dropped, logging:
`Inactive WhatsApp channel: unknown - <phone_number>.`
The fix extends `get_channel_from_wb_payload` to fall back to normalized
phone number matching using the existing PhoneNumberNormalizationService
normalizers (Brazil, Argentina), which were previously only used for
contact-level lookups.
---------
Co-authored-by: Sojan Jose <sojan@pepalo.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Muhsin <12408980+muhsin-k@users.noreply.github.com>
WhatsApp contacts using coexistence are identified by more than one
source ID (a phone `wa_id` and a `BR.`/BSUID identity), so a single
contact ends up owning multiple `contact_inbox` records. The "reopen the
same conversation" feature scoped conversation reuse to a single
`contact_inbox`, so messages arriving under a different identity of the
same contact started a brand-new conversation — even with reopen enabled
— producing duplicate conversations.
This scopes reuse to the contact across all of its `contact_inbox`
records in the inbox instead of a single `contact_inbox`.
## Closes
- [CW-7651
](https://linear.app/chatwoot/issue/CW-7651/duplicate-conversations)
## How to reproduce
1. On a WhatsApp Cloud inbox with "reopen the same conversation" (lock
to single conversation) enabled.
2. Have a coexistence contact whose webhooks alternate between carrying
the phone `wa_id` and only the BSUID identity.
3. Before: each identity opens its own conversation → duplicates. After:
incoming messages reopen the contact's existing conversation regardless
of which identity the webhook carried.
## What changed
- `Whatsapp::IncomingMessageBaseService#set_conversation` now looks up
reusable conversations via `@contact.conversations.where(inbox_id:
@inbox.id)` instead of `@contact_inbox.conversations`.
- Updated existing specs to wire the conversation's `contact` to the
contact_inbox's contact, mirroring production data.
## Description
Adds an API-only branded email layout feature for Email inbox replies.
Administrators can configure an account-level fallback layout and
per-email-inbox overrides with Liquid HTML using `{{ content_for_layout
}}`, and eligible outbound email replies/transcripts render through the
scoped layout when the account feature flag `branded_email_templates` is
enabled.
The feature is disabled by default and is manually controlled through
the normal account feature flag mechanism.
Fixes
https://linear.app/chatwoot/issue/CW-7514/branded-html-email-templates-per-inboxbrand
## Type of change
- [x] New feature (non-breaking change which adds functionality)
- [x] This change requires a documentation update
## How to test
1. Start Chatwoot locally and sign in as an administrator.
2. Enable the account feature flag for the account you are testing:
```ruby
account = Account.find(<account_id>)
account.enable_features!(:branded_email_templates)
```
3. Create or pick an Email inbox, then note the `account_id` and
`inbox_id`.
4. Configure an account-level fallback layout through the API using
authenticated admin headers:
```http
PATCH /api/v1/accounts/:account_id/branded_email_layout
Content-Type: application/json
{
"branded_email_layout": "<html><body><header>Account Brand</header>{{
content_for_layout }}<footer>Account footer</footer></body></html>"
}
```
5. Confirm `GET /api/v1/accounts/:account_id/branded_email_layout`
returns the saved account layout.
6. Configure an inbox-level override for the Email inbox:
```http
PATCH /api/v1/accounts/:account_id/inboxes/:inbox_id
Content-Type: application/json
{
"branded_email_layout": "<html><body><header>Inbox Brand</header>{{
content_for_layout }}<footer>Inbox footer</footer></body></html>"
}
```
7. Confirm `GET /api/v1/accounts/:account_id/inboxes/:inbox_id` returns
the inbox `branded_email_layout`.
8. Send an Email inbox reply and verify the outbound email body is
wrapped with the inbox layout around the generated reply content.
9. Clear the inbox layout by sending a blank value, then send another
reply and verify it falls back to the account layout:
```http
PATCH /api/v1/accounts/:account_id/inboxes/:inbox_id
Content-Type: application/json
{
"branded_email_layout": ""
}
```
10. Clear the account layout with a blank value and verify Email replies
return to the existing no-layout behavior.
11. Verify validation behavior:
- Updating either API with a layout that omits `{{ content_for_layout
}}` returns `422`.
- Updating either API with invalid Liquid returns `422`.
- Updating a non-Email inbox with `branded_email_layout` returns `422`.
- Disabling `branded_email_templates` and updating a layout returns
`422`.
## How Has This Been Tested?
Validation:
- `bundle exec rspec spec/models/email_template_spec.rb
spec/controllers/api/v1/accounts/branded_email_layouts_controller_spec.rb
spec/controllers/api/v1/accounts/inboxes_controller_spec.rb
spec/lib/email_templates/db_resolver_service_spec.rb
spec/mailers/conversation_reply_mailer_spec.rb`
- `bundle exec rspec
spec/enterprise/services/enterprise/billing/handle_stripe_event_service_spec.rb
spec/enterprise/services/internal/reconcile_plan_config_service_spec.rb`
- `bundle exec rubocop` on changed Ruby files, excluding generated
`db/schema.rb`
- `git diff --check` and `git diff --cached --check`
- YAML parsing for changed config/Swagger files
- `bundle exec rails routes -g branded_email_layout`
## Checklist:
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have made corresponding changes to the documentation
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have commented on my code, particularly in hard-to-understand
areas
- [ ] My changes generate no new warnings
- [ ] Any dependent changes have been merged and published in downstream
modules
The `[WHATSAPP_MANUAL_TRANSFER] success` log introduced in #14975 to
track embedded signup → manual migrations was firing on any WhatsApp
credential change — including routine `api_key` rotations on inboxes
that were already manually configured — inflating the migration count.
The log now fires only for the actual migration, and uses a new tag so
log searches don't match the older over-counted entries.
## How to reproduce
1. On a manually configured WhatsApp Cloud inbox, update the API key
from inbox settings → Configuration.
2. Before this change, the app log records a `[WHATSAPP_MANUAL_TRANSFER]
success` line even though no migration happened; after this change it
stays silent.
3. Switching an embedded signup inbox to manual setup still logs the
migration (now as `[WHATSAPP_EMBEDDED_TO_MANUAL] success`).
## What changed
- `Channel::Whatsapp#log_credentials_transfer` now keys off the
migration's unique signal — `provider_config['source']` changing from
`embedded_signup` to absent — instead of diffing credential keys.
- Renamed the log tag from `WHATSAPP_MANUAL_TRANSFER` to
`WHATSAPP_EMBEDDED_TO_MANUAL` (success and failure lines) so the
corrected entries are searchable without matching pre-fix false
positives.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Muhsin <12408980+muhsin-k@users.noreply.github.com>
Sony Mathew