## Linear task:
https://linear.app/chatwoot/issue/CW-6318/searchkickimporterror-type-=-mapper-parsing-exception-reason-=-failed
## Description
Fixes Elasticsearch `mapper_parsing_exception` errors that occur when
`campaign_id` contain non-numeric string values
## Type of change
- [x] Bug fix (non-breaking change which fixes an issue)
## How Has This Been Tested?
- Unit tests
- use a local OpenSearch 3.4.0 cluster to verify actual indexing
behavior.
## Checklist:
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged and published in downstream
modules
<!-- CURSOR_SUMMARY -->
---
> [!NOTE]
> Removes `campaign_id` from the message search index payload to
simplify `additional_attributes`, keeping only `automation_rule_id`.
>
> - `Messages::SearchDataPresenter#additional_attributes_data` now
returns only `automation_rule_id`
> - Specs updated to stop asserting `campaign_id` and continue
validating `automation_rule_id` and email subject handling
>
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
5a9c8eb794a044e3f258b644f67a6731de9e904c. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
- Fix ordered lists being sent as unordered lists in WhatsApp
integrations
- The WhatsApp markdown renderer was converting all lists to bullet
points (-), ignoring numbered list formatting
- Added ordered list support by tracking list_type and list_item_number
from CommonMarker AST metadata
Before:
Input: "1. First\n2. Second\n3. Third"
Output: "- First\n- Second\n- Third"
After:
Input: "1. First\n2. Second\n3. Third"
Output: "1. First\n2. Second\n3. Third"
Fixes https://github.com/chatwoot/chatwoot/issues/13317
Fixes an issue where WhatsApp attachment messages (images, audio, video,
documents) were failing to download. Messages were being created but
without attachments.
The `phone_number_id` parameter was being passed to the `GET
/<MEDIA_ID>` endpoint when downloading incoming media. According to
Meta's documentation:
> "Note that `phone_number_id` is optional. If included, the request
will only be processed if the business phone number ID included in the
query matches the ID of the business
phone number **that the media was uploaded on**."
For incoming messages, media is uploaded by the customer, not by the
business phone number. Passing the business's `phone_number_id` causes
validation to fail with error: `Param phone_number_id is not a valid
whatsapp business phone number id ID`
This PR removes the `phone_number_id` parameter from the media URL
request for incoming messages.
Fixes https://github.com/chatwoot/chatwoot/issues/13097
### Problem
The PR #12176 removed the `before_save :setup_webhooks` callback to fix
a race condition where Meta's webhook verification request arrived
before the channel was saved to the database. This change broke manual
WhatsApp Cloud channel setup. While embedded signup explicitly calls
`channel.setup_webhooks` in `EmbeddedSignupService`, manual setup had no
equivalent call - meaning the `subscribed_apps` endpoint was never
invoked and Meta never sent webhook events to Chatwoot.
### Solution
Added an `after_commit` callback that triggers webhook setup for manual
WhatsApp Cloud channels
Ensure CSAT survey label rules are evaluated once in CsatSurveyService
before any channel-specific sending (including WhatsApp/Twilio
templates), remove the duplicated rule check from the template builder,
and cover the blocking-label scenario in service specs while simplifying
the template specs accordingly.
Co-authored-by: Sojan Jose <sojan@pepalo.com>
## Linear Ticket
https://linear.app/chatwoot/issue/CW-4569/nomethoderror-undefined-method-blocked-for-nil-nomethoderror
## Description
Fixes NoMethodError in ConversationMuteHelpers that occurs during
contact deletion race condition.
When a contact is deleted, there's a brief window (~50-150ms) where
contact_id becomes nil but conversations still exist. If ResolutionJob
runs during this window, the muted? method crashes trying to call
blocked? on nil.Fixes # (issue)
## Type of change
- [ ] Bug fix (non-breaking change which fixes an issue)
## How Has This Been Tested?
- Created orphaned conversations (contact_id = nil)
- Called muted?, mute!, unmute! - all return gracefully
- Verified async deletion still works correctly
## Checklist:
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged and published in downstream
modules
Co-authored-by: Sojan Jose <sojan@pepalo.com>
CSAT scores are helpful, but on their own they rarely tell the full
story. A drop in rating can come from delayed timelines, unclear
expectations, or simple misunderstandings, even when the issue itself
was handled correctly.
Review Notes for CSAT let admins/report manager roles add internal-only
context next to each CSAT response. This makes it easier to interpret
scores properly and focus on patterns and root causes, not just numbers.
<img width="2170" height="1680" alt="image"
src="https://github.com/user-attachments/assets/56df7fab-d0a7-4a94-95b9-e4c459ad33d5"
/>
### Why this matters
* Capture the real context behind individual CSAT ratings
* Clarify whether a low score points to a genuine service issue or a
process gap
* Spot recurring themes across conversations and teams
* Make CSAT reviews more useful for leadership reviews and
retrospectives
### How Review Notes work
**View CSAT responses**
Open the CSAT report to see overall metrics, rating distribution, and
individual responses.
**Add a Review Note**
For any CSAT entry, managers can add a Review Note directly below the
customer’s feedback.
**Document internal insights**
Use Review Notes to capture things like:
* Why a score was lower or higher than expected
* Patterns you are seeing across similar cases
* Observations around communication, timelines, or customer expectations
Review Notes are visible only to administrators and people with report
access only. We may expand visibility to agents in the future based on
feedback. However, customers never see them.
Each note clearly shows who added it and when, making it easy to review
context and changes over time.
This API gives you how many conversations exist per channel, broken down
by status in a given time period. The max time period is capped to 6
months for now.
**Input Params:**
- **since:** Unix timestamp (seconds) - start of date range
- **until:** Unix timestamp (seconds) - end of date range
**Response Payload:**
```json
{
"Channel::Sms": {
"resolved": 85,
"snoozed": 10,
"open": 5,
"pending": 5,
"total": 100
},
"Channel::Email": {
"resolved": 72,
"snoozed": 15,
"open": 13,
"pending": 13,
"total": 100
},
"Channel::WebWidget": {
"resolved": 90,
"snoozed": 7,
"open": 3,
"pending": 3,
"total": 100
}
}
```
**Definitons:**
resolved = Number of conversations created within the selected time
period that are currently marked as resolved.
snoozed = Number of conversations created within the selected time
period that are currently marked as snoozed.
pending = Number of conversations created within the selected time
period that are currently marked as pending.
open = Number of conversations created within the selected time period
that are currently open.
total = Total number of conversations created within the selected time
period, across all statuses.
https://one.newrelic.com/alerts/issue?account=3437125&duration=259200000&state=d088e9b7-d0ce-3fcf-fda5-145df8b9cb2a
## Description
Pass serialized data instead of ActiveRecord object in
dispatch_destroy_event to prevent ActiveJob::DeserializationError when
the notification is already deleted.
This error occurs frequently because RemoveDuplicateNotificationJob
deletes notifications, and by the time the async EventDispatcherJob
runs, the record no longer exists.
## Type of change
- [ ] Bug fix (non-breaking change which fixes an issue)
## How Has This Been Tested?
## Checklist:
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged and published in downstream
modules
<!-- CURSOR_SUMMARY -->
---
> [!NOTE]
> Avoids ActiveJob deserialization failures by sending serialized data
for notification deletion and updating the listener accordingly.
>
> - `Notification#dispatch_destroy_event` now dispatches
`NOTIFICATION_DELETED` with serialized `notification_data` (`id`,
`user_id`, `account_id`) instead of the AR object
> - `ActionCableListener#notification_deleted` reads
`notification_data`, finds `User`/`Account`, computes
`unread_count`/`count` via `NotificationFinder`, and broadcasts using
the user’s pubsub token
> - Specs updated to pass `notification_data` and assert payload
(including `unread_count`/`count`)
>
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
e2ffbe765b148fdfd2cd2e031c657c36e423c1f5. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
---------
Co-authored-by: Vishnu Narayanan <iamwishnu@gmail.com>
## Description
Adds the ability to sort companies by the number of contacts they have
(contacts_count) in ascending or descending order. This is part of the
Chatwoot 5.0 release requirements for the companies feature.
The implementation uses a scope-based approach consistent with other
sorting implementations in the codebase (e.g., contacts sorting by
last_activity_at).
## Type of change
- [x] New feature (non-breaking change which adds functionality)
## Available Sorting Options
After this change, the Companies API supports the following sorting
options:
| Sort Field | Type | Ascending | Descending |
|------------|------|-----------|------------|
| `name` | string | `?sort=name` | `?sort=-name` |
| `domain` | string | `?sort=domain` | `?sort=-domain` |
| `created_at` | datetime | `?sort=created_at` | `?sort=-created_at` |
| `contacts_count` | integer (scope) | `?sort=contacts_count` |
`?sort=-contacts_count` |
**Note:** Prefix with `-` for descending order. Companies with NULL
contacts_count will appear last (NULLS LAST).
## CURL Examples
**Sort by contacts count (ascending):**
```bash
curl -X GET 'https://app.chatwoot.com/api/v1/accounts/{account_id}/companies?sort=contacts_count' \
-H 'api_access_token: YOUR_API_TOKEN'
```
**Sort by contacts count (descending):**
```bash
curl -X GET 'https://app.chatwoot.com/api/v1/accounts/{account_id}/companies?sort=-contacts_count' \
-H 'api_access_token: YOUR_API_TOKEN'
```
**Sort by name (ascending):**
```bash
curl -X GET 'https://app.chatwoot.com/api/v1/accounts/{account_id}/companies?sort=name' \
-H 'api_access_token: YOUR_API_TOKEN'
```
**Sort by created_at (descending):**
```bash
curl -X GET 'https://app.chatwoot.com/api/v1/accounts/{account_id}/companies?sort=-created_at' \
-H 'api_access_token: YOUR_API_TOKEN'
```
**With pagination:**
```bash
curl -X GET 'https://app.chatwoot.com/api/v1/accounts/{account_id}/companies?sort=-contacts_count&page=2' \
-H 'api_access_token: YOUR_API_TOKEN'
```
## How Has This Been Tested?
- Added RSpec tests for both ascending and descending sort
- All 24 existing specs pass
- Manually tested the sorting functionality with test data
**Test configuration:**
- Ruby 3.4.4
- Rails 7.1.5.2
- PostgreSQL (test database)
**To reproduce:**
1. Run `bundle exec rspec
spec/enterprise/controllers/api/v1/accounts/companies_controller_spec.rb`
2. All tests should pass (24 examples, 0 failures)
## Checklist:
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
## Technical Details
**Backend changes:**
- Controller: Added `sort_on :contacts_count` with scope-based sorting
- Model: Added `order_on_contacts_count` scope using
`Arel::Nodes::SqlLiteral` and `sanitize_sql_for_order` with `NULLS LAST`
for consistent NULL handling
- Specs: Added 2 new tests for ascending/descending sort validation
**Files changed:**
- `enterprise/app/controllers/api/v1/accounts/companies_controller.rb`
- `enterprise/app/models/company.rb`
-
`spec/enterprise/controllers/api/v1/accounts/companies_controller_spec.rb`
**Note:** This PR only includes the backend implementation. Frontend
changes (sort menu UI + i18n) will follow in a separate commit.
---------
Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
Co-authored-by: iamsivin <iamsivin@gmail.com>
Co-authored-by: Sivin Varghese <64252451+iamsivin@users.noreply.github.com>
Co-authored-by: Pranav <pranav@chatwoot.com>
## Description
Implements comprehensive search functionality with advanced filtering
capabilities for Chatwoot (Linear: CW-5956).
This PR adds:
1. **Time-based filtering** for contacts and conversations (SQL-based
search)
2. **Advanced message search** with multiple filters
(OpenSearch/Elasticsearch-based)
- **`from` filter**: Filter messages by sender (format: `contact:42` or
`agent:5`)
- **`inbox_id` filter**: Filter messages by specific inbox
- **Time range filters**: Filter messages using `since` and `until`
parameters (Unix timestamps in seconds)
- **90-day limit enforcement**: Automatically limits searches to the
last 90 days to prevent performance issues
The implementation extends the existing `Enterprise::SearchService`
module for advanced features and adds time filtering to the base
`SearchService` for SQL-based searches.
## API Documentation
### Base URL
All search endpoints follow this pattern:
```
GET /api/v1/accounts/{account_id}/search/{resource}
```
### Authentication
All requests require authentication headers:
```
api_access_token: YOUR_ACCESS_TOKEN
```
---
## 1. Search All Resources
**Endpoint:** `GET /api/v1/accounts/{account_id}/search`
Returns results from all searchable resources (contacts, conversations,
messages, articles).
### Parameters
| Parameter | Type | Description | Required |
|-----------|------|-------------|----------|
| `q` | string | Search query | Yes |
| `page` | integer | Page number (15 items per page) | No |
| `since` | integer | Unix timestamp (contacts/conversations only) | No
|
| `until` | integer | Unix timestamp (contacts/conversations only) | No
|
### Example Request
```bash
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search?q=customer" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
### Example Response
```json
{
"payload": {
"contacts": [...],
"conversations": [...],
"messages": [...],
"articles": [...]
}
}
```
---
## 2. Search Contacts
**Endpoint:** `GET /api/v1/accounts/{account_id}/search/contacts`
Search contacts by name, email, phone number, or identifier with
optional time filtering.
### Parameters
| Parameter | Type | Description | Required |
|-----------|------|-------------|----------|
| `q` | string | Search query | Yes |
| `page` | integer | Page number (15 items per page) | No |
| `since` | integer | Unix timestamp - filter by last_activity_at | No |
| `until` | integer | Unix timestamp - filter by last_activity_at | No |
### Example Requests
**Basic search:**
```bash
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/contacts?q=john" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
**Search contacts active in the last 7 days:**
```bash
SINCE=$(date -v-7d +%s)
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/contacts?q=john&since=${SINCE}" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
**Search contacts active between 30 and 7 days ago:**
```bash
SINCE=$(date -v-30d +%s)
UNTIL=$(date -v-7d +%s)
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/contacts?q=john&since=${SINCE}&until=${UNTIL}" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
### Example Response
```json
{
"payload": {
"contacts": [
{
"id": 42,
"email": "john@example.com",
"name": "John Doe",
"phone_number": "+1234567890",
"identifier": "user_123",
"additional_attributes": {},
"created_at": 1701234567
}
]
}
}
```
---
## 3. Search Conversations
**Endpoint:** `GET /api/v1/accounts/{account_id}/search/conversations`
Search conversations by display ID, contact name, email, phone number,
or identifier with optional time filtering.
### Parameters
| Parameter | Type | Description | Required |
|-----------|------|-------------|----------|
| `q` | string | Search query | Yes |
| `page` | integer | Page number (15 items per page) | No |
| `since` | integer | Unix timestamp - filter by last_activity_at | No |
| `until` | integer | Unix timestamp - filter by last_activity_at | No |
### Example Requests
**Basic search:**
```bash
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/conversations?q=billing" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
**Search conversations active in the last 24 hours:**
```bash
SINCE=$(date -v-1d +%s)
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/conversations?q=billing&since=${SINCE}" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
**Search conversations from last month:**
```bash
SINCE=$(date -v-30d +%s)
UNTIL=$(date +%s)
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/conversations?q=billing&since=${SINCE}&until=${UNTIL}" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
### Example Response
```json
{
"payload": {
"conversations": [
{
"id": 123,
"display_id": 45,
"inbox_id": 1,
"status": "open",
"messages": [...],
"meta": {...}
}
]
}
}
```
---
## 4. Search Messages (Advanced)
**Endpoint:** `GET /api/v1/accounts/{account_id}/search/messages`
Advanced message search with multiple filters powered by
OpenSearch/Elasticsearch.
### Prerequisites
- OpenSearch/Elasticsearch must be running (`OPENSEARCH_URL` env var
configured)
- Account must have `advanced_search` feature flag enabled
- Messages must be indexed in OpenSearch
### Parameters
| Parameter | Type | Description | Required |
|-----------|------|-------------|----------|
| `q` | string | Search query | Yes |
| `page` | integer | Page number (15 items per page) | No |
| `from` | string | Filter by sender: `contact:{id}` or `agent:{id}` |
No |
| `inbox_id` | integer | Filter by specific inbox ID | No |
| `since` | integer | Unix timestamp - searches from this time (max 90
days ago) | No |
| `until` | integer | Unix timestamp - searches until this time | No |
### Important Notes
- **90-Day Limit**: If `since` is not provided, searches default to the
last 90 days
- If `since` exceeds 90 days, returns `422` error: "Search is limited to
the last 90 days"
- All time filters use message `created_at` timestamp
### Example Requests
**Basic message search:**
```bash
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
**Search messages from a specific contact:**
```bash
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund&from=contact:42" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
**Search messages from a specific agent:**
```bash
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund&from=agent:5" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
**Search messages in a specific inbox:**
```bash
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund&inbox_id=3" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
**Search messages from the last 7 days:**
```bash
SINCE=$(date -v-7d +%s)
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund&since=${SINCE}" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
**Search messages between specific dates:**
```bash
SINCE=$(date -v-30d +%s)
UNTIL=$(date -v-7d +%s)
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund&since=${SINCE}&until=${UNTIL}" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
**Combine all filters:**
```bash
SINCE=$(date -v-14d +%s)
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund&from=contact:42&inbox_id=3&since=${SINCE}" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
**Attempt to search beyond 90 days (returns error):**
```bash
SINCE=$(date -v-120d +%s)
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund&since=${SINCE}" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
### Example Response (Success)
```json
{
"payload": {
"messages": [
{
"id": 789,
"content": "I need a refund for my purchase",
"message_type": "incoming",
"created_at": 1701234567,
"conversation_id": 123,
"inbox_id": 3,
"sender": {
"id": 42,
"type": "contact"
}
}
]
}
}
```
### Example Response (90-day limit exceeded)
```json
{
"error": "Search is limited to the last 90 days"
}
```
**Status Code:** `422 Unprocessable Entity`
---
## 5. Search Articles
**Endpoint:** `GET /api/v1/accounts/{account_id}/search/articles`
Search help center articles by title or content.
### Parameters
| Parameter | Type | Description | Required |
|-----------|------|-------------|----------|
| `q` | string | Search query | Yes |
| `page` | integer | Page number (15 items per page) | No |
### Example Request
```bash
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/articles?q=installation" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
### Example Response
```json
{
"payload": {
"articles": [
{
"id": 456,
"title": "Installation Guide",
"slug": "installation-guide",
"portal_slug": "help",
"account_id": 1,
"category_name": "Getting Started",
"status": "published",
"updated_at": 1701234567
}
]
}
}
```
---
## Technical Implementation
### SQL-Based Search (Contacts, Conversations, Articles)
- Uses PostgreSQL `ILIKE` queries by default
- Optional GIN index support via `search_with_gin` feature flag for
better performance
- Time filtering uses `last_activity_at` for contacts/conversations
- Returns paginated results (15 per page)
### Advanced Search (Messages)
- Powered by OpenSearch/Elasticsearch via Searchkick gem
- Requires `OPENSEARCH_URL` environment variable
- Requires `advanced_search` account feature flag
- Enforces 90-day lookback limit via
`Limits::MESSAGE_SEARCH_TIME_RANGE_LIMIT_DAYS`
- Validates inbox access permissions before filtering
- Returns paginated results (15 per page)
---
## Type of change
- [x] New feature (non-breaking change which adds functionality)
- [x] Enhancement (improves existing functionality)
---
## How Has This Been Tested?
### Unit Tests
- **Contact Search Tests**: 3 new test cases for time filtering
(`since`, `until`, combined)
- **Conversation Search Tests**: 3 new test cases for time filtering
- **Message Search Tests**: 10+ test cases covering:
- Individual filters (`from`, `inbox_id`, time range)
- Combined filters
- Permission validation for inbox access
- Feature flag checks
- 90-day limit enforcement
- Error handling for exceeded time limits
### Test Commands
```bash
# Run all search controller tests
bundle exec rspec spec/controllers/api/v1/accounts/search_controller_spec.rb
# Run search service tests (includes enterprise specs)
bundle exec rspec spec/services/search_service_spec.rb
```
### Manual Testing Setup
A rake task is provided to create 50,000 test messages across multiple
inboxes:
```bash
# 1. Create test data
bundle exec rake search:setup_test_data
# 2. Start OpenSearch
mise elasticsearch-start
# 3. Reindex messages
rails runner "Message.search_index.import Message.all"
# 4. Enable feature flag
rails runner "Account.first.enable_features('advanced_search')"
# 5. Test via API or Rails console
```
---
## Checklist
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented on my code, particularly in hard-to-understand
areas
- [x] I have made corresponding changes to the documentation (this PR
description)
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged and published in downstream
modules
---
## Additional Notes
### Requirements
- **OpenSearch/Elasticsearch**: Required for advanced message search
- Set `OPENSEARCH_URL` environment variable
- Example: `export OPENSEARCH_URL=http://localhost:9200`
- **Feature Flags**:
- `advanced_search`: Account-level flag for message advanced search
- `search_with_gin` (optional): Account-level flag for GIN-based SQL
search
### Performance Considerations
- 90-day limit prevents expensive long-range queries on large datasets
- GIN indexes recommended for high-volume search on SQL-based resources
- OpenSearch/Elasticsearch provides faster full-text search for messages
### Breaking Changes
- None. All new parameters are optional and backward compatible.
### Frontend Integration
- Frontend PR tracking advanced search UI will consume these endpoints
- Time range pickers should convert JavaScript `Date` to Unix timestamps
(seconds)
- Date conversion: `Math.floor(date.getTime() / 1000)`
### Error Handling
- Invalid `from` parameter format is silently ignored (filter not
applied)
- Time range exceeding 90 days returns `422` with error message
- Missing `q` parameter returns `422` (existing behavior)
- Unauthorized inbox access is filtered out (no error, just excluded
from results)
---------
Co-authored-by: iamsivin <iamsivin@gmail.com>
Co-authored-by: Sivin Varghese <64252451+iamsivin@users.noreply.github.com>
Co-authored-by: Pranav <pranav@chatwoot.com>
Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
When AgentBot responds to customer messages, the `waiting_since`
timestamp is not reset, causing inflated reply time metrics when a human
agent eventually responds. This results in inaccurate reporting that
incorrectly includes periods when customers were satisfied with bot
responses.
### Timeline from Production Data
```
Dec 12, 16:20:14 - Customer sends message (ID: 368451924)
↓ waiting_since = Dec 12, 16:20:14
Dec 12, 16:20:17 - AgentBot replies (ID: 368451960)
↓ waiting_since STILL = Dec 12, 16:20:14 ❌
↓ (Bot response doesn't clear it)
14-day gap - Customer satisfied, no messages
↓ waiting_since STILL = Dec 12, 16:20:14 ❌
Dec 26, 22:25:45 - Customer sends new message (ID: 383522275)
↓ waiting_since STILL = Dec 12, 16:20:14 ❌
↓ (New message doesn't reset it)
Dec 26-27 - More AgentBot interactions
↓ waiting_since STILL = Dec 12, 16:20:14 ❌
Dec 27, 07:36:53 - Human agent finally replies (ID: 383799517)
↓ Reply time calculated: 1,268,404 seconds
↓ = 14.7 DAYS ❌
```
## Root Cause
The core issues is in `app/models/message.rb`, where **AgentBot messages
does not clear `waiting_since`** - The `human_response?` method only
returns true for `User` senders, so bot replies never trigger the
clearing logic. This means once `waiting_since` is set, it stays set
even when customers send new messages after receiving bot responses.
The solution is to simply reset `waiting_since` **after a bot has
responded**. This ensures reply time metrics reflect actual human agent
response times, not bot-handled periods.
### What triggers the rest
This is an intentional "gotcha", that only `AgentBot` and
`Captain::Assistant` messages trigger the waiting time reset. Automation
and campaign messages maintain current behavior (no reset). This is
because interactive bot assistants provide conversational help that
might satisfy customers. Automation and campaigns are one-way
communications and shouldn't affect waiting time calculations.
## Related Work
Extends PR #11787 which fixed `waiting_since` clearing on conversation
resolution. This PR addresses the bot interaction scenario which was not
covered by that fix.
Scripts to clean data:
https://gist.github.com/scmmishra/bd133208e219d0ab52fbfdf03036c48a
Problem: SystemStackError: stack level too deep occurred when rendering
messages with indented content (4+ spaces) for WhatsApp, Instagram, and
Facebook channels.
Root Cause: CommonMarker::Renderer#code_block contains a self-recursive
placeholder that must be overridden:
```
def code_block(node)
code_block(node) # calls itself infinitely
end
```
WhatsAppRenderer and InstagramRenderer were missing this override,
causing infinite recursion when markdown with 4-space indentation
(interpreted as code blocks) was rendered.
Fix: Added code_block method to both renderers that outputs the node
content as plain text:
```
def code_block(node)
out(node.string_content)
end
```
Fix https://linear.app/chatwoot/issue/CW-6217/systemstackerror-stack-level-too-deep-systemstackerror
Previously, attachments relied only on blob_id, which made it possible
to attach blobs across accounts by enumerating IDs. We now require both
blob_id and blob_key, add cross-account validation to prevent blob
reuse, and centralize the logic in a shared BlobOwnershipValidation
concern.
It also fixes a frontend bug where mixed-type action params (number +
string) were incorrectly dropped, causing attachment uploads to fail.
fixes: #11834
This pull request introduces TikTok channel integration, enabling users
to connect and manage TikTok business accounts similarly to other
supported social channels. The changes span backend API endpoints,
authentication helpers, webhook handling, configuration, and frontend
components to support TikTok as a first-class channel.
**Key Notes**
* This integration is only compatible with TikTok Business Accounts
* Special permissions are required to access the TikTok [Business
Messaging
API](https://business-api.tiktok.com/portal/docs?id=1832183871604753).
* The Business Messaging API is region-restricted and is currently
unavailable to users in the EU.
* Only TEXT, IMAGE, and POST_SHARE messages are currently supported due
to limitations in the TikTok Business Messaging API
* A message will be successfully sent only if it contains text alone or
one image attachment. Messages with multiple attachments or those
combining text and attachments will fail and receive a descriptive error
status.
* Messages sent directly from the TikTok App will be synced into the
system
* Initiating a new conversation from the system is not permitted due to
limitations from the TikTok Business Messaging API.
**Backend: TikTok Channel Integration**
* Added `Api::V1::Accounts::Tiktok::AuthorizationsController` to handle
TikTok OAuth authorization initiation, returning the TikTok
authorization URL.
* Implemented `Tiktok::CallbacksController` to handle TikTok OAuth
callback, process authorization results, create or update channel/inbox,
and handle errors or denied scopes.
* Added `Webhooks::TiktokController` to receive and verify TikTok
webhook events, including signature verification and event dispatching.
* Created `Tiktok::IntegrationHelper` module for JWT-based token
generation and verification for secure TikTok OAuth state management.
**Configuration and Feature Flags**
* Added TikTok app credentials (`TIKTOK_APP_ID`, `TIKTOK_APP_SECRET`) to
allowed configs and app config, and registered TikTok as a feature in
the super admin features YAML.
[[1]](diffhunk://#diff-5e46e1d248631a1147521477d84a54f8ba6846ea21c61eca5f70042d960467f4R43)
[[2]](diffhunk://#diff-8bf37a019cab1dedea458c437bd93e34af1d6e22b1672b1d43ef6eaa4dcb7732R69)
[[3]](diffhunk://#diff-123164bea29f3c096b0d018702b090d5ae670760c729141bd4169a36f5f5c1caR74-R79)
**Frontend: TikTok Channel UI and Messaging Support**
* Added `TiktokChannel` API client for frontend TikTok authorization
requests.
* Updated channel icon mappings and tests to include TikTok
(`Channel::Tiktok`).
[[1]](diffhunk://#diff-b852739ed45def61218d581d0de1ba73f213f55570aa5eec52aaa08f380d0e16R16)
[[2]](diffhunk://#diff-3cd3ae32e94ef85f1f2c4435abf0775cc0614fb37ee25d97945cd51573ef199eR64-R69)
* Enabled TikTok as a supported channel in contact forms, channel
widgets, and feature toggles.
[[1]](diffhunk://#diff-ec59c85e1403aaed1a7de35971fe16b7033d5cd763be590903ebf8f1ca25a010R47)
[[2]](diffhunk://#diff-ec59c85e1403aaed1a7de35971fe16b7033d5cd763be590903ebf8f1ca25a010R69)
[[3]](diffhunk://#diff-725b90ca7e3a6837ec8291e9f57094f6a46b3ee00e598d16564f77f32cf354b0R26-R29)
[[4]](diffhunk://#diff-725b90ca7e3a6837ec8291e9f57094f6a46b3ee00e598d16564f77f32cf354b0R51-R54)
[[5]](diffhunk://#diff-725b90ca7e3a6837ec8291e9f57094f6a46b3ee00e598d16564f77f32cf354b0R68)
* Updated message meta logic to support TikTok-specific message statuses
(sent, delivered, read).
[[1]](diffhunk://#diff-e41239cf8dda36c1bd1066dbb17588ae8868e56289072c74b3a6d7ef5abdd696R23)
[[2]](diffhunk://#diff-e41239cf8dda36c1bd1066dbb17588ae8868e56289072c74b3a6d7ef5abdd696L63-R65)
[[3]](diffhunk://#diff-e41239cf8dda36c1bd1066dbb17588ae8868e56289072c74b3a6d7ef5abdd696L81-R84)
[[4]](diffhunk://#diff-e41239cf8dda36c1bd1066dbb17588ae8868e56289072c74b3a6d7ef5abdd696L103-R107)
* Added support for embedded message attachments (e.g., TikTok embeds)
with a new `EmbedBubble` component and updated message rendering logic.
[[1]](diffhunk://#diff-c3d701caf27d9c31e200c6143c11a11b9d8826f78aa2ce5aa107470e6fdb9d7fR31)
[[2]](diffhunk://#diff-047859f9368a46d6d20177df7d6d623768488ecc38a5b1e284f958fad49add68R1-R19)
[[3]](diffhunk://#diff-c3d701caf27d9c31e200c6143c11a11b9d8826f78aa2ce5aa107470e6fdb9d7fR316)
[[4]](diffhunk://#diff-cbc85e7c4c8d56f2a847d0b01cd48ef36e5f87b43023bff0520fdfc707283085R52)
* Adjusted reply policy and UI messaging for TikTok's 48-hour reply
window.
[[1]](diffhunk://#diff-0d691f6a983bd89502f91253ecf22e871314545d1e3d3b106fbfc76bf6d8e1c7R208-R210)
[[2]](diffhunk://#diff-0d691f6a983bd89502f91253ecf22e871314545d1e3d3b106fbfc76bf6d8e1c7R224-R226)
These changes collectively enable end-to-end TikTok channel support,
from configuration and OAuth flow to webhook processing and frontend
message handling.
------------
# TikTok App Setup & Configuration
1. Grant access to the Business Messaging API
([Documentation](https://business-api.tiktok.com/portal/docs?id=1832184145137922))
2. Set the app authorization redirect URL to
`https://FRONTEND_URL/tiktok/callback`
3. Update the installation config with TikTok App ID and Secret
4. Create a Business Messaging Webhook configuration and set the
callback url to `https://FRONTEND_URL/webhooks/tiktok`
([Documentation](https://business-api.tiktok.com/portal/docs?id=1832190670631937))
. You can do this by calling
`Tiktok::AuthClient.update_webhook_callback` from rails console once you
finish Tiktok channel configuration in super admin ( will be automated
in future )
5. Enable TikTok channel feature in an account
---------
Co-authored-by: Sojan Jose <sojan@pepalo.com>
Co-authored-by: iamsivin <iamsivin@gmail.com>
- Linear::CallbacksController: Replace broken
`described_class.new`mocking with proper `GlobalConfigService` stubbing
and real JWT token generation. The old pattern doesn't work in request
specs since Rails instantiates controllers internally.
- User associations: Remove `.class_name('Conversation')` assertion that
fails intermittently due to enterprise `prepend_mod_with` timing in
parallel tests. The class_name is already enforced by Rails at runtime -
if wrong, the app would crash immediately. No need to explicitly test
for this
Fixes
https://linear.app/chatwoot/issue/CW-6138/debug-linear-and-user-spec-failures-in-ci
## Summary
Fixes the issue where double newlines (paragraph breaks) were collapsing
to single newlines in text-based messaging channels (Telegram, WhatsApp,
Instagram, Facebook, LINE, SMS).
### Root Cause
The `preserve_multiple_newlines` method only preserved 3+ consecutive
newlines using the regex `/\n{3,}/`. When users pressed Enter twice
(creating a paragraph break with 2 newlines), CommonMarker would parse
this as separate paragraphs, which then collapsed to a single newline in
the output.
This caused:
- ❌ Normal Enter: Double newlines collapsed to single newline
- ✅ Shift+Enter: Worked (created hard breaks)
### Fix
Changed the regex from `/\n{3,}/` to `/\n{2,}/` to preserve 2+
consecutive newlines. This prevents CommonMarker from collapsing
paragraph breaks.
Now:
- ✅ Single newline (`\n`) → Single newline (handled by softbreak)
- ✅ Double newline (`\n\n`) → Double newline (preserved with
placeholders)
- ✅ Triple+ newlines → Preserved as before
### Test Coverage
Added comprehensive tests for:
- Single newlines preservation
- Double newlines (paragraph breaks) preservation
- Multiple consecutive newlines
- Newlines with varying amounts of whitespace between them (1 space, 3
spaces, 5 spaces, tabs)
All 66 tests passing.
### Impact
This fix affects all text-based messaging channels that use the markdown
renderer:
- Telegram
- WhatsApp
- Instagram
- Facebook
- LINE
- SMS
- Twilio SMS (when configured for WhatsApp)
Fixes
https://linear.app/chatwoot/issue/CW-6135/double-newline-is-breaking
---------
Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
## Description
Fixes an issue where multiple newlines with whitespace between them
(e.g., `\n \n \n`) were being collapsed to single newlines in text-based
messaging channels (Telegram, WhatsApp, Instagram, Facebook, Line, SMS).
The frontend was sending messages with spaces/tabs between newlines, and
the markdown renderer was treating these as paragraph content,
collapsing them during rendering.
### Changes:
1. Added whitespace normalization in `render_telegram_html`,
`render_whatsapp`, `render_instagram`, `render_line`, and
`render_plain_text` methods
2. Strips whitespace from whitespace-only lines before markdown
processing
3. Added comprehensive regression tests for all affected channels
## Type of change
- [x] Bug fix (non-breaking change which fixes an issue)
## How Has This Been Tested?
1. **Unit Tests**: Added 7 new specs testing multiple newlines with
whitespace between them for all text-based channels
2. **Manual Testing**: Verified with actual frontend payload containing
`\n \n \n` patterns
3. **Regression Testing**: All existing 63 specs pass
### Test Results:
- ✅ All 63 markdown renderer specs pass (56 original + 7 new)
- ✅ All 12 Telegram channel specs pass
- ✅ All 27 WhatsApp + Instagram specs pass
- ✅ Verified with real-world payload: 18 newlines preserved (previously
collapsed to 1)
### Test Command:
```bash
RAILS_ENV=test bundle exec rspec spec/services/messages/markdown_renderer_service_spec.rb
RAILS_ENV=test bundle exec rspec spec/models/channel/telegram_spec.rb
```
## Checklist:
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective
- [x] New and existing unit tests pass locally with my changes
Sojan Jose