## Description
Refactored the `Message#search_data` method to prevent exceeding
OpenSearch's 1000 field limit during reindex operations.
**Problem:** The previous implementation serialized entire ActiveRecord
objects (Inbox, Sender, Conversation) with all their JSONB fields,
causing dynamic field explosion in OpenSearch. This resulted in
`Searchkick::ImportError` with "Limit of total fields [1000] has been
exceeded".
**Solution:** Whitelisted only necessary fields for search and
filtering, and flattened JSONB `custom_attributes` into key-value pair
arrays to prevent unbounded field creation.
Linked to: CW-5861
## Type of change
- [x] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality not to work as expected)
- [x] This change requires a documentation update
## How Has This Been Tested?
- Verified rubocop passes with no offenses
- Code review of search field usage from
`enterprise/app/services/enterprise/search_service.rb`
- Analyzed actual search queries to determine required indexed fields
**Still needed:**
- Full reindex test on staging/production environment
- Verify search functionality still works after reindex
- Confirm field count is under 1000 limit
## Changes Made
### Before
- Indexed 1000+ fields (entire AR objects with JSONB)
- `inbox` = full Inbox object (23+ fields + JSONB)
- `sender` = full Contact/User/AgentBot object (10+ fields + JSONB)
- `conversation` = full push_event_data
- Dynamic JSONB keys creating unlimited fields
### After
- ~35-40 controlled fields
- Whitelisted search fields: `content`, `attachment_transcribed_text`,
`email_subject`
- Filter fields: `account_id`, `inbox_id`, `conversation_id`,
`sender_id`, `sender_type`, etc.
- Flattened `custom_attributes`: `[{key, value, value_type}]` format
- Helper methods: `search_conversation_data`, `search_inbox_data`,
`search_sender_data`, `search_additional_data`
## Checklist:
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged and published in downstream
modules
## Post-merge Steps
After merging, the following steps are required:
1. **Reindex all messages:**
```bash
bundle exec rails runner "Message.reindex"
```
2. **Verify field count:**
```bash
bundle exec rails runner "
client = Searchkick.client
index_name = Message.searchkick_index.name
mapping = client.indices.get_mapping(index: index_name)
fields = mapping.dig(index_name, 'mappings', 'properties')
puts 'Total fields: ' + fields.keys.count.to_s
"
```
3. **Test search functionality** to ensure queries still work as
expected
---------
Co-authored-by: Vishnu Narayanan <iamwishnu@gmail.com>
Co-authored-by: Pranav <pranav@chatwoot.com>
# Pull Request Template
## Description
This PR makes the contacts bulk action bar sticky while scrolling.
## Type of change
- [x] Bug fix (non-breaking change which fixes an issue)
## How Has This Been Tested?
### Screenshots
<img width="1080" height="300" alt="image"
src="https://github.com/user-attachments/assets/21f8f3c6-813e-4ef6-b40a-8dd14e6ffb26"
/>
<img width="1080" height="300" alt="image"
src="https://github.com/user-attachments/assets/bb939f1d-9a13-4f9f-953d-b9872c984b74"
/>
## Checklist:
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [ ] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged and published in downstream
modules
# Pull Request Template
## Description
**This PR includes:**
1. Added URL-based filter persistence for the responses pages, including
page and search parameters.
2. Introduced a new empty state variant for pending FAQs — without a
backdrop and with a “Clear Filters” option.
3. Made the actions, filter, and search row remain fixed at the top
while scrolling.
Fixes
https://linear.app/chatwoot/issue/CW-5852/improvements-in-pending-faqs
## Type of change
- [x] New feature (non-breaking change which adds functionality)
## How Has This Been Tested?
### Loom video
https://www.loom.com/share/1d9eee68c0684f0ab05e08b4ca1e0ce9
## Checklist:
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [ ] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged and published in downstream
modules
# Pull Request Template
## Description
**This PR includes,**
- Added new pending FAQs view with approve/edit/delete actions for each
response.
- Implemented banner notification showing pending FAQ count on main
approved responses page.
- Created dedicated route for pending FAQs review at
/captain/responses/pending.
- Added automatic pending count updates when switching assistants or
routes.
- Modified ResponseCard component to show action buttons instead of
dropdown in pending view.
Fixes
https://linear.app/chatwoot/issue/CW-5833/pending-faqs-in-a-different-ux
## Type of change
- [x] New feature (non-breaking change which adds functionality)
## How Has This Been Tested?
### Loom video
https://www.loom.com/share/5fe8f79b04cd4681b9360c48710b9373
## Checklist:
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged and published in downstream
modules
---------
Co-authored-by: Pranav <pranav@chatwoot.com>
### Problem
WhatsApp Cloud channels already handle Brazil/Argentina phone number
format mismatches (PRs #12492, #11173), but Twilio WhatsApp channels
were creating duplicate contacts
when:
- Template sent to new format: `whatsapp:+5541988887777` (13 digits)
- User responds from old format: `whatsapp:+554188887777` (12 digits)
### Solution
The solution extends the existing phone number normalization
infrastructure to support both WhatsApp providers while handling their
different payload formats:
### Provider Format Differences
- **WhatsApp Cloud**: `wa_id: "919745786257"` (clean number)
- **Twilio WhatsApp**: `From: "whatsapp:+919745786257"` (prefixed
format)
### Test Coverage
#### Brazil Phone Number Tests
**Case 1: New Format (13 digits with "9")**
- **Test 1**: No existing contact → Creates new contact with original
format
- **Test 2**: Contact exists in same format → Appends to existing
conversation
**Case 2: Old Format (12 digits without "9")**
- **Test 3**: Contact exists in old format → Appends to existing
conversation
- **Test 4** *(Critical)*: Contact exists in new format, message in old
format → Finds existing contact, prevents duplicate
- **Test 5**: No contact exists → Creates new contact with incoming
format
#### Argentina Phone Number Tests
**Case 3: With "9" after country code**
- **Test 6**: No existing contact → Creates new contact
- **Test 7**: Contact exists in normalized format → Uses existing
contact
**Case 4: Without "9" after country code**
- **Test 8**: Contact exists in same format → Appends to existing
- **Test 9**: No contact exists → Creates new contact
Fixes
https://linear.app/chatwoot/issue/CW-5565/inconsistencies-for-mobile-numbersargentina-brazil-and-mexico-numbers
# Pull Request Template
## Description
This PR introduces a new changelog component that can be used in the
sidebar.
Fixes
https://linear.app/chatwoot/issue/CW-5776/changelog-card-ui-component
## Type of change
- [x] New feature (non-breaking change which adds functionality)
## How Has This Been Tested?
### Screencast
https://github.com/user-attachments/assets/42e77e82-388a-4fc9-9b37-f3d0ea1a9d7f
## Checklist:
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged and published in downstream
modules
---------
Co-authored-by: Muhsin <muhsinkeramam@gmail.com>
# Pull Request Template
## Description
This PR enables active tabs in inbox settings to persist in the URL for
easy sharing.
## Type of change
- [x] New feature (non-breaking change which adds functionality)
## How Has This Been Tested?
### Loom video
https://www.loom.com/share/63820ecb17ea491a9082339f8bb457b6?sid=4fef1acd-b4fd-431f-855c-7647015a330f
## Checklist:
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged and published in downstream
modules
---------
Co-authored-by: Muhsin <muhsinkeramam@gmail.com>
# Pull Request Template
## Description
This PR enables URL-based search and tab selection, allowing search
queries and active tabs to persist in the URL for easy sharing.
Fixes
[CW-5766](https://linear.app/chatwoot/issue/CW-5766/cannot-impersonate-an-account),
https://github.com/chatwoot/chatwoot/issues/12623
## Type of change
- [x] New feature (non-breaking change which adds functionality)
## How Has This Been Tested?
### Loom video
https://www.loom.com/share/422a1d61f3fe4278a88e352ef98d2b78?sid=35fabee7-652f-4e17-83bd-e066a3bb804c
## Checklist:
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged and published in downstream
modules
We should avoid using margins to define space between elements, instead
use the gap utility.
The problem with this particular instance was that if Google auth was
turned off and SSO is available, there is a weird spacing at the top
caused by the margin from the SSO element.
This PR will fix that. It also introduces a gap between the divider and
the button, but that should be okay.
# Pull Request Template
## Description
This PR removes the frame containing all channel icons from the “Create
Inbox” page.
## Type of change
- [x] Bug fix (non-breaking change which fixes an issue)
## How Has This Been Tested?
### Screenshots
**Before**
<img width="1314" height="1016" alt="image"
src="https://github.com/user-attachments/assets/2b773495-9ddb-48b4-b15d-9aef18259ce1"
/>
**After**
<img width="1314" height="979" alt="image"
src="https://github.com/user-attachments/assets/f4dc64cf-516c-4faf-a45c-2f7de05cc29b"
/>
## Checklist:
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [ ] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged and published in downstream
modules
## Context
Sidekiq logs only showed the Sidekiq wrapper class and JID, which wasn’t
helpful when debugging ActiveJobs.
## Changes
- Updated `ChatwootDequeuedLogger` to log the actual `ActiveJob class`
and `job_id` instead of the generic Sidekiq wrapper and JID.
> Example
> ```
> Dequeued ActionMailer::MailDeliveryJob
123e4567-e89b-12d3-a456-426614174000 from default
> ```
- Remove sidekiq worker and unify everything to `ActiveJob`
Previously, email replies were handled inside workers. There was no
execution logs. This meant if emails silently failed (as reported by a
customer), we had no way to trace where the issue happened, the only
assumption was “no error = mail sent.”
By moving email handling into jobs, we now have proper execution logs
for each attempt. This makes it easier to debug delivery issues and
would have better visibility when investigating customer reports.
Fixes
https://linear.app/chatwoot/issue/CW-5538/emails-are-not-sentdelivered-to-the-contact
---------
Co-authored-by: Sojan Jose <sojan@pepalo.com>
Co-authored-by: Shivam Mishra <scm.mymail@gmail.com>
This PR collapses multiple queries fetching stats from a single table to
a single query
```sql
SELECT
user_id as user_id,
COUNT(CASE WHEN name = 'conversation_resolved' THEN 1 END) as resolved_count,
AVG(CASE WHEN name = 'conversation_resolved' THEN value END) as avg_resolution_time,
AVG(CASE WHEN name = 'first_response' THEN value END) as avg_first_response_time,
AVG(CASE WHEN name = 'reply_time' THEN value END) as avg_reply_time
FROM "reporting_events"
WHERE
"reporting_events"."account_id" = <account_id> AND
"reporting_events"."created_at" >= '2025-09-14 18:30:00' AND
"reporting_events"."created_at" < '2025-10-14 18:29:59'
GROUP BY "reporting_events"."user_id";
```
### Why this works?
Here's why this optimization is faster based on PostgreSQL internals:
- Single Table Scan vs Multiple Scans: Earlier we did 4 sequential scans
(or 4 index scans) of the same data, with the same where clause, now in
a single scan all 4 `CASE` expressions are evaluated in a single pass.
- Shared Buffer Cache Efficiency: PostgreSQL's shared buffer cache
stores recently accessed pages, with this, pages are loaded once and
re-used for all aggregation, earlier with separate queries we were
forced to re-read all from the disk each time
- Reduced planning and network overhead (4 vs 1 query)
### How is it tested
1. The specs all pass without making any changes
2. Verified the reports side by side after generating from report seeder
#### How to test
Generate seed data using the following command
```bash
ACCOUNT_ID=1 ENABLE_ACCOUNT_SEEDING=true bundle exec rake db:seed:reports_data
```
Once done download the reports, checkout to this branch and download the
reports again and compare them
This PR adds inbox filtering to the conversation traffic heatmap,
allowing users to analyze patterns for specific inboxes. Additionally,
it also adds a new resolution count heatmap that shows when support
teams are most active in resolving conversations, using a green color to
distinguish it from the blue conversation heatmap.
The PR also reorganizes heatmap components into a cleaner structure with
a shared `BaseHeatmapContainer` that handles common functionality like
date range selection, inbox filtering, and data fetching. This makes it
easy to add new heatmap metrics in the future - just create a wrapper
component specifying the metric type and color scheme.
<img width="1926" height="1670" alt="CleanShot 2025-10-13 at 14 01
35@2x"
src="https://github.com/user-attachments/assets/67822a34-6170-4d19-9e11-7ad4ded5c388"
/>
<img width="1964" height="1634" alt="CleanShot 2025-10-13 at 14 03
00@2x"
src="https://github.com/user-attachments/assets/e4613c08-64b8-4fa6-91d8-7510946dd75d"
/>
Unrelated change, the data seeder conversation resolution would not work
correctly, we've fixed it.
---------
Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
## Changelog
- Added conditional Active Record encryption to every external
credential we store (SMTP/IMAP passwords, Twilio tokens,
Slack/OpenAI hook tokens, Facebook/Instagram tokens, LINE/Telegram keys,
Twitter secrets) so new writes are encrypted
whenever Chatwoot.encryption_configured? is true; legacy installs still
receive plaintext until their secrets are
updated.
- Tuned encryption settings in config/application.rb to allow legacy
reads (support_unencrypted_data) and to extend
deterministic queries so lookups continue to match plaintext rows during
the rollout; added TODOs to retire the
fallback once encryption becomes mandatory.
- Introduced an MFA-pipeline test suite
(spec/models/external_credentials_encryption_spec.rb) plus shared
examples to
verify each attribute encrypts at rest and that plaintext records
re-encrypt on update, with a dedicated Telegram case.
The existing MFA GitHub workflow now runs these tests using the
preconfigured encryption keys.
fixes:
https://linear.app/chatwoot/issue/CW-5453/encrypt-sensitive-credentials-stored-in-plain-text-in-database
## Testing Instructions
1. Instance without encryption keys
- Unset ACTIVE_RECORD_ENCRYPTION_* vars (or run in an environment where
they’re absent).
- Create at least one credentialed channel (e.g., Email SMTP).
- Confirm workflows still function (send/receive mail or a similar
sanity check).
- In the DB you should still see plaintext values—this confirms the
guard prevents encryption when keys are missing.
2. Instance with encryption keys
- Configure the three encryption env vars and restart.
- Pick a couple of representative integrations (e.g., Email SMTP +
Twilio SMS).
- Legacy channel check:
- Use existing records created before enabling keys. Trigger their
workflow (send an email / SMS, or hit the
webhook) to ensure they still authenticate.
- Inspect the raw column—value remains plaintext until changed.
- Update legacy channel:
- Edit one legacy channel’s credential (e.g., change SMTP password).
- Verify the operation still works and the stored value is now encrypted
(raw column differs, accessor returns
original).
- New channel creation:
- Create a new channel of the same type; confirm functionality and that
the stored credential is encrypted from
the start.
---------
Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
# Changelog
When an agent bot webhook fails, we now flip any pending conversation
back to an open state so a human agent can pick
it up immediately. There will be an clear activity message giving the
team clear visibility into what went
wrong. This keeps customers from getting stuck in limbo when their
connected bot goes offline.
# Testing instructions
1. Initial setup: Create an agent bot with a working webhook URL and
connect it to a test inbox. Send a message from a
contact (e.g., via the widget) so a conversation is created; it should
enter the Pending state while the bot handles
the reply.
2. Introduce failure: Edit that agent bot and swap the webhook URL for a
dummy endpoint that will fail. Have the same
contact send another message in the existing conversation. Because the
webhook call now fails, the conversation should flip from Pending back
to Open, making it visible to agents. Also verify the activity message
3. New conversation check: With the dummy URL still in place, start a
brand-new conversation from a contact. When the
bot tries (and fails) to respond, confirm that the conversation appears
immediately as Open rather than remaining Pending. Also the activity
message is visible
4. Subsequent messages in open conversations will show no change
---------
Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
Tanmay Deep Sharma