## Description
Reduces sustained load during large Intercom imports by limiting
conversation list pages to 10 while retaining 50-contact pages. Cursor
and provider total-count behavior remain unchanged.
The one-minute progress heartbeat now lives in parent PR #15050 because
stalled-import detection must be safe when that PR is deployed
independently. This child PR therefore contains only the smaller-page
delta.
This is Phase 1, Task 2 of the [Intercom import optimization plan
(CW-7615)](https://linear.app/chatwoot/issue/CW-7615/optimize-intercom-import-reliability-and-bulk-message-ingestion).
This PR is stacked on #15050 and should be reviewed as the two-file
delta from `codex/cw-7519-intercom-stalled-retry-15m`.
## Closes
-
[CW-7615](https://linear.app/chatwoot/issue/CW-7615/optimize-intercom-import-reliability-and-bulk-message-ingestion)
## Type of change
- [x] Bug fix (non-breaking change which fixes an issue)
## How to test
1. Start an Intercom import containing contacts and conversations across
multiple source pages.
2. Confirm contact list requests retain a page size of 50.
3. Confirm conversation list requests use a page size of 10.
4. Confirm page cursors and displayed provider totals continue advancing
normally.
5. Confirm the parent PR heartbeat keeps long conversations fresh while
these shorter pages are processed.
## Checklist
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have added tests that prove the change is effective
- [x] New and existing focused tests pass locally with my changes
## Description
Prepares each Intercom conversation's source message and parts as one
deterministic batch before persistence. The builder preserves provider
order, prefetches mappings and live messages once, and classifies every
entry as current-run, previous-run, stale-mapping repair,
existing-message repair, or new.
The importer consumes those prepared entries through the existing
individual transaction and retry path, so activity conversion, private
notes, sender attribution, metadata, timestamps, skip logs, idempotency,
indexing, and heartbeat behavior remain unchanged. This PR intentionally
does not bulk insert messages; batching writes is Phase 2 Task 2.
This is Phase 2, Task 1 (Task 4 overall) of
[CW-7615](https://linear.app/chatwoot/issue/CW-7615/optimize-intercom-import-reliability-and-bulk-message-ingestion).
This PR is stacked on #15052 and should be reviewed as the four-file
delta from `codex/cw-7615-intercom-query-timeout-retries`.
## Closes
- Tracking plan:
[CW-7615](https://linear.app/chatwoot/issue/CW-7615/optimize-intercom-import-reliability-and-bulk-message-ingestion)
## Type of change
- [x] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality not to work as expected)
- [ ] This change requires a documentation update
## How to test
1. Import a conversation containing a source message, comments, notes,
activities, and entries with equal timestamps.
2. Retry the same run and verify messages and counters remain
idempotent.
3. Retry after a previous run and verify existing mappings are reported
as skips without changing their owner.
4. Delete a mapped Message, rerun, and verify the message and mapping
are repaired.
5. Delete only a mapping, rerun, and verify the existing Message is
reused without duplication.
6. Rerun a previously skipped part that now classifies as an activity
and verify it is repaired.
7. Import an empty conversation and verify no message or mapping
prefetch is performed.
## Checklist
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged and published in downstream
modules
## Description
Reduces database pressure during large Intercom imports by deferring
same-run contact and message statistic recounts until the page boundary,
immediately before the cursor advances. Repeated mappings within a page
now mark their statistic group dirty instead of running aggregate counts
and persisting statistics for every record.
Contact and message database attempts now retry once when PostgreSQL
cancels a query. The failed transaction is allowed to roll back before
the importer waits for a randomized 200-500 ms delay and retries. A
recovered timeout creates no error; a second timeout follows the
existing item/message failure path exactly once. Provider, validation,
and other database errors are not retried.
This is Phase 1, Task 3 of the [Intercom import optimization plan
(CW-7615)](https://linear.app/chatwoot/issue/CW-7615/optimize-intercom-import-reliability-and-bulk-message-ingestion).
This PR is stacked on #15051 and should be reviewed as the two-file
delta from `codex/cw-7519-intercom-shorter-jobs-heartbeats`.
## Closes
- Tracking plan:
[CW-7615](https://linear.app/chatwoot/issue/CW-7615/optimize-intercom-import-reliability-and-bulk-message-ingestion)
## Type of change
- [x] Bug fix (non-breaking change which fixes an issue)
## How to test
1. Retry a page whose contacts or messages already have same-run
mappings and confirm imported/skipped statistics are reconciled before
the page cursor advances.
2. Raise one `ActiveRecord::QueryCanceled` while persisting a contact
mapping and confirm the transaction rolls back, retries, and creates one
contact and mapping without an error.
3. Repeat for a message mapping and confirm one message and mapping are
created without an error.
4. Raise the timeout on both attempts and confirm one final item/message
error is recorded.
5. Raise an unrelated database error and confirm it is not retried.
## Checklist
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have added tests that prove the change is effective
- [x] New and existing focused tests pass locally with my changes
This preserves per-message failure isolation when an Intercom bulk
message chunk cannot be committed. Query timeouts are retried once after
the failed transaction rolls back, while other database failures move
directly to an entry-by-entry fallback.
The fallback reacquires the import lock and refreshes each entry before
writing, so a competing worker cannot create duplicate messages. Each
individual write uses a savepoint, and search indexing happens only
after the import lock transaction commits.
This is Phase 2, Task 6 of CW-7615 and is stacked on #15111. The review
delta is one commit across the importer and its focused spec.
## Closes
-
[CW-7615](https://linear.app/chatwoot/issue/CW-7615/optimize-intercom-import-reliability-and-bulk-message-ingestion)
## How to test
1. Run an Intercom import where a bulk mapping write times out once and
confirm the chunk succeeds on retry.
2. Force a persistent timeout or database write failure and confirm
valid messages are imported through the fallback.
3. Force one fallback record to fail and confirm only that message is
recorded as an import error.
4. Simulate another worker repairing a message before fallback and
confirm no duplicate is created.
## Description
Adds an admin-only Intercom import workflow under Settings > Data.
Admins can connect an Intercom access token, start named historical
contact/conversation imports, monitor active and previous import runs,
review paginated skip/error logs, download skip logs, and route imported
conversations into source-bucket API inboxes that can be renamed later.
The import path stores durable source mappings, batches Intercom
contact/conversation pages through Sidekiq, records already-imported
records as skipped, and writes historical messages without normal
outbound delivery callbacks. The PR also includes the Intercom import
PRD/TDD document for review context.
Closes
[CW-7519](https://linear.app/chatwoot/issue/CW-7519/explore-intercom-import)
## Type of change
- [ ] Bug fix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality not to work as expected)
- [x] This change requires a documentation update
## How Has This Been Tested?
Tested importing using actual data through integration.
Screenshots:
<img width="1800" height="948" alt="Screenshot 2026-07-02 at 10 48
48 PM"
src="https://github.com/user-attachments/assets/e74d9ed6-0bca-47de-b6ef-e589afcddfde"
/>
<img width="1800" height="1008" alt="Screenshot 2026-07-02 at 10 49
03 PM"
src="https://github.com/user-attachments/assets/1bd12fdb-0a47-4287-ac1d-ea308e70a9cd"
/>
<img width="1800" height="1005" alt="Screenshot 2026-07-02 at 10 49
21 PM"
src="https://github.com/user-attachments/assets/3d8145f5-1794-4cc3-b3fa-de5cd80e6ca3"
/>
<img width="1800" height="1002" alt="Screenshot 2026-07-02 at 10 49
38 PM"
src="https://github.com/user-attachments/assets/6f818efd-4193-43c2-84eb-66970dca4490"
/>
Passed locally:
```sh
eval "$(rbenv init -)" && bundle exec rspec spec/models/data_import_spec.rb spec/jobs/data_import_job_spec.rb spec/requests/api/v1/accounts/data_imports_spec.rb spec/requests/api/v1/accounts/integrations/intercom_spec.rb spec/jobs/data_imports/intercom/import_jobs_spec.rb spec/services/data_imports/intercom/importer_spec.rb spec/services/data_imports/intercom/placeholder_inbox_builder_spec.rb spec/services/data_imports/intercom/source_bucket_spec.rb
```
```sh
eval "$(rbenv init -)" && bundle exec rubocop app/controllers/api/v1/accounts/data_imports_controller.rb app/controllers/api/v1/accounts/integrations/intercom_controller.rb app/jobs/data_imports/intercom app/models/data_import.rb app/models/data_import_error.rb app/models/data_import_item.rb app/models/data_import_mapping.rb app/models/integrations/hook.rb app/policies/data_import_policy.rb app/policies/hook_policy.rb app/services/data_imports/intercom db/migrate/20260702000000_expand_data_imports_for_intercom_imports.rb db/migrate/20260702000001_create_data_import_items.rb db/migrate/20260702000002_create_data_import_mappings.rb db/migrate/20260702000003_create_data_import_errors.rb spec/jobs/data_imports/intercom spec/requests/api/v1/accounts/data_imports_spec.rb spec/requests/api/v1/accounts/integrations/intercom_spec.rb spec/services/data_imports/intercom
```
```sh
pnpm exec eslint app/javascript/dashboard/api/dataImports.js app/javascript/dashboard/api/integrations.js app/javascript/dashboard/routes/dashboard/settings/data/Index.vue app/javascript/dashboard/routes/dashboard/settings/data/Show.vue app/javascript/dashboard/routes/dashboard/settings/data/data.routes.js app/javascript/dashboard/routes/dashboard/settings/data/importStatus.js app/javascript/dashboard/routes/dashboard/settings/integrations/Intercom.vue app/javascript/dashboard/routes/dashboard/settings/integrations/integrations.routes.js app/javascript/dashboard/routes/dashboard/settings/settings.routes.js app/javascript/dashboard/components-next/sidebar/Sidebar.vue app/javascript/dashboard/routes/dashboard/settings/inbox/Index.vue
```
```sh
git diff --check
```
Note: the RSpec boot logs the existing local `chatwoot_dev` purge
warning because other database sessions are open, then continues and
completes with 52 examples, 0 failures.
## Checklist:
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [ ] I have commented on my code, particularly in hard-to-understand
areas
- [x] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged and published in downstream
modules
---------
Co-authored-by: Shivam Mishra <scm.mymail@gmail.com>
Co-authored-by: Sivin Varghese <64252451+iamsivin@users.noreply.github.com>
Co-authored-by: iamsivin <iamsivin@gmail.com>
Sony Mathew