17 Commits
Author SHA1 Message Date
Aakash BakhleandGitHub 9c68eed676 fix(captain): send custom tool API key headers (#15008)
Captain custom tools configured with API key authentication now send the
configured key in the requested HTTP header. Existing tools begin
working without needing to be recreated or reconfigured, while
credentials remain protected across redirects.

## How to reproduce

1. Create a Captain custom tool using API Key authentication.
2. Configure `X-API-Key` as the header name and save the tool.
3. Invoke the tool and inspect the incoming request.
4. Before this change, the API key header is absent; after this change,
the configured endpoint receives it.

## What changed

The UI persists API key authentication as `name` and `key`, but the
request builder also required an unused `location: header` property. The
request builder now treats API key authentication as header-based,
matching the only mode exposed by the UI.

Custom authentication headers are also registered as sensitive with
`SafeFetch`. They are retained for the configured endpoint and
same-origin redirects, but stripped when a redirect crosses origins to
prevent credential leakage. Factory, request, and redirect specs cover
the real UI payload and both public and private-network fetch paths.
2026-07-15 22:43:30 +05:30
Aakash BakhleandGitHub fd625981e9 fix(captain): keep custom tools available in Captain V2 (#15015)
Captain V2 assistants can now use every enabled custom tool from their
account through the main assistant. The change keeps existing custom
tool access when an assistant has no migrated scenarios, so switching
from V1 does not remove the capability without warning.

## How to reproduce

1. Create and enable an account custom tool.
2. Use an assistant with no custom instructions and no generated
scenarios.
3. Enable Captain V2 for the account.
4. Before this change, the main assistant receives only FAQ lookup and
handoff. After this change, it also receives the enabled account custom
tool.

## What changed

The main V2 assistant now loads enabled custom tools through its account
association. Scenario agents still load only the tools named in their
scenario instructions. The account custom tool limit keeps the added
tool count bounded.

Focused model coverage verifies enabled tools, disabled tools, account
isolation, FAQ lookup, and handoff. Existing V1 assistant, V2 scenario,
and V2 runner coverage passes. RuboCop passes.
2026-07-15 11:35:57 +05:30
df7f137657 feat: add captain sessions model [CW-7485] (#14970)
This adds a `captain_sessions` table to log every Captain run, starting
with Assistant Responses and Copilot Responses. Each session records the
assistant, model, credits consumed, the FAQs/documents/scenario that
contributed to the response, and the full run context — giving customers
visibility into how a response was generated and giving us durable stats
on credit, FAQ, and document usage (which today only exist as ephemeral
trace metadata and an aggregate account counter).

## What changed

- New `Captain::Session` model with a `session_type` enum (`assistant`,
`copilot`). The subject (`Conversation` / `CopilotThread`) and result
(`Message` / `CopilotMessage`) classes are inferred from the session
type, so the table stores plain `subject_id` / `result_id` ids.
`result_id` is nullable so failed runs that still consumed credits can
be logged.
- Composite indexes on `[session_type, subject_id]`, `[session_type,
result_id]`, and `[account_id, session_type, created_at]` for lookup and
usage-stats queries.
- Factory and model specs.

This PR is schema + model only; the writer/instrumentation that records
sessions from the assistant and copilot flows will follow.

---------

Co-authored-by: Sony Mathew <sony@chatwoot.com>
2026-07-13 18:18:20 +05:30
6a262287d2 feat(captain): allow agents to report Captain messages (#14799)
Adds a cloud-only flow for agents to flag incorrect or problematic
Captain (AI) responses. Right-clicking a Captain message surfaces a
"Report message" option that opens a dialog to pick a problem type and
add a description, persisted to a new captain_message_reports table for
the team to review.


<img width="636" height="542" alt="Screenshot 2026-06-20 at 9 15 56 AM"
src="https://github.com/user-attachments/assets/afaa233d-6bd6-455a-8a33-a3796a3e3ef6"
/>
<img width="580" height="502" alt="Screenshot 2026-06-20 at 9 16 03 AM"
src="https://github.com/user-attachments/assets/2d220d99-98cc-4c5e-a325-778ceb4f7bc9"
/>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 14:47:08 -07:00
Aakash BakhleandGitHub 2182165201 feat: sync documents job [AI-142] (#14057)
# Pull Request Template

## Description

Document auto-sync job pipeline without wiring to controllers or FE

Fixes # (issue)

## Type of change

- [x] New feature (non-breaking change which adds functionality)

## How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide
instructions so we can reproduce. Please also list any relevant details
for your test configuration.
specs and locally

## Checklist:

- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] Any dependent changes have been merged and published in downstream
modules
2026-04-22 23:36:32 +05:30
dbe35252bc fix: Use handoff_key for scenarios (#13755)
# Pull Request Template

## Description

Ensure agent function names stay within OpenAI's 64-char limit
(ai-agents prepends "handoff_to_").

Add HANDOFF_TITLE_SLUG_MAX_LENGTH and
handoff_key generation: persisted records use `scenario_{id}_agent`; new
records use a truncated title slug.
Assistant scenario keys and agent_name now reference the generated
handoff key.

fixes :

`Invalid 'messages[9].tool_calls[0].function.name': string too long.
Expected a string with maximum length 64, but got a string with length
95 instead.`

## Type of change


- [x] Bug fix (non-breaking change which fixes an issue)

## How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide
instructions so we can reproduce. Please also list any relevant details
for your test configuration.

Tested locally
<img width="1806" height="1044" alt="image"
src="https://github.com/user-attachments/assets/40cd7a3d-3d97-43a8-bd56-d3f5d63abbda"
/>

## Checklist:

- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] Any dependent changes have been merged and published in downstream
modules

---------

Co-authored-by: Shivam Mishra <scm.mymail@gmail.com>
2026-03-11 14:01:25 +05:30
7c60ad9e28 feat: include contact verified status with each tool call (#13663)
Co-authored-by: aakashb95 <aakashbakhle@gmail.com>
2026-02-26 16:16:33 +05:30
Sojan JoseandGitHub d2ba9a2ad3 feat(enterprise): add voice conference API (#13064)
The backend APIs for the voice call channel 
ref: #11602
2025-12-15 15:11:59 -08:00
Shivam MishraandGitHub 0d8e249fe4 feat: include chatwoot metadata with each tool call (#12907) 2025-12-10 15:25:18 +05:30
Vishnu NarayananandGitHub 08b9134486 feat: speed up circleci and github actions (#12849)
# 🚀 Speed up CI/CD test execution with parallelization

## TL;DR

- **Problem**: CI tests took 36-42 minutes per commit, blocking
developer workflow
- **Solution**: Implemented 16-way parallelization + optimized slow
tests + fixed Docker builds
- **Impact**: **1,358 hours/month saved** (7.7 FTE) across GitHub
Actions + CircleCI
  - GitHub Actions tests: 36m → 7m (82% faster)
  - Backend tests: 28m → 4m (87% faster with 16-way parallelization)
  - CircleCI tests: 42m → 7m (83% faster)
  - Docker builds: 34m → 5m (85% faster)
- **Result**: 5-6x faster feedback loops, 100% success rate on recent
runs

---

## Problem
CI test runs were taking **36-42 minutes per commit push** (GitHub
Actions: 36m avg, CircleCI: 42m P95), creating a significant bottleneck
in the development workflow.

## Solution
This PR comprehensively restructures both CI test pipelines to leverage
16-way parallelization and optimize test execution, reducing test
runtime from **36-42 minutes to ~7 minutes** - an **82% improvement**.

---

## 📊 Real Performance Data (Both CI Systems)

### GitHub Actions

#### Before (develop branch - 5 recent runs)
```
Individual runs: 35m 29s | 36m 1s | 40m 0s | 36m 4s | 34m 18s
Average: 36m 22s
```

#### After (feat/speed_up_ci branch - 9 successful runs)
```
Individual runs: 6m 39s | 7m 2s | 6m 53s | 6m 26s | 6m 52s | 6m 42s | 6m 45s | 6m 40s | 6m 37s
Average: 6m 44s
Range: 6m 26s - 7m 2s
```

**Improvement**:  **81.5% faster** (29m 38s saved per run)


#### Backend Tests Specific Impact
With 16-way parallelization, backend tests show dramatic improvement:
- **Before**: 27m 52s (sequential execution)
- **After**: 3m 44s (longest of 16 parallel runners)
  - Average across runners: 2m 30s
  - Range: 1m 52s - 3m 44s
- **Improvement**:  **86.6% faster** (24m 8s saved)
---

### CircleCI

#### Before (develop branch - CircleCI Insights)
```
Duration (P95): 41m 44s
Runs: 70 (last 30 days)
Success Rate: 84%
```

#### After (feat/speed_up_ci branch - Last 2 pipeline runs)
```
Run 1 (1h ago): 7m 7s
  ├─ lint: 4m 12s
  ├─ frontend-tests: 5m 36s
  ├─ backend-tests: 6m 23s
  ├─ coverage: 20s
  └─ build: 1s

Run 2 (2h ago): 7m 21s
  ├─ lint: 3m 47s
  ├─ frontend-tests: 5m 4s
  ├─ backend-tests: 6m 33s
  ├─ coverage: 19s
  └─ build: 1s

Average: 7m 14s
Success Rate: 100% 
```

**Improvement**:  **82.7% faster** (34m 30s saved per run)

---

## 🐳 Related Work: Docker Build Optimization

As part of the broader CI/CD optimization effort, Docker build
performance was improved separately in **PR #12859**.

### Docker Build Fix (Merged Separately)
**Problem**: Multi-architecture Docker builds (amd64/arm64) were taking
~34 minutes due to cache thrashing

**Solution**: Added separate cache scopes per platform in
`.github/workflows/test_docker_build.yml`:
```yaml
cache-from: type=gha,scope=${{ matrix.platform }}
cache-to: type=gha,mode=max,scope=${{ matrix.platform }}
```

**Results** (measured from November 2025 data):
- **Before**: 34.2 minutes/run average (15,547 minutes across 454 runs)
- **After**: 5 minutes/run
- **Improvement**: 85% faster, 29.2 minutes saved per run
- **Frequency**: 25.2 runs/day
- **Monthly savings**: **369 hours** (46 developer-days)

This prevents different architectures from invalidating each other's
caches and contributes 27% of total CI/CD time savings.

---

## 🎯 Key Findings

### Both CI Systems Now Perform Similarly
- **CircleCI**: 7m 14s average
- **GitHub Actions**: 6m 44s average
- **Difference**: Only 30 seconds apart (remarkably consistent!)

### Combined Performance
- **Average improvement across both systems**: **82.1% faster**
- **Time saved per commit**: ~32 minutes
- **Developer feedback loop**: 36-42 minutes → ~7 minutes

### Success Rate Improvement
- **CircleCI**: 84% → 100% (on feat/speed_up_ci branch)
- **GitHub Actions**: 100% (all 9 recent runs successful)
- Fixed all test isolation issues that caused intermittent failures

### Impact at Scale (Based on Real November 2025 Data)
- **CI runs per day**: **30.8 average** for tests, **25.2** for Docker
builds
  - Measured from GitHub Actions Usage Metrics (18 days)
  - Weekdays: 38-54 runs/day
  - Peak: up to 68 runs in a single day
- **This PR (test suite only)**:
  - **Daily time saved**: **15.3 hours** (GitHub Actions + CircleCI)
- **Monthly time saved**: **458 hours** (57 developer-days) on GitHub
Actions
  - Additional **531 hours** (66 developer-days) on CircleCI
- **Combined with Docker optimization** (PR #12859): **1,358
hours/month** (see Summary)
- **Developer experience**: 5-6x faster iteration cycles

---

## Code Changes

### 1. **Backend Test Parallelization (16x)**
Both CI systems now use 16-way parallelization with identical
round-robin test distribution:

```bash
# Distribute tests evenly across 16 runners
SPEC_FILES=($(find spec -name '*_spec.rb' | sort))
for i in "${!SPEC_FILES[@]}"; do
  if [ $(( i % 16 )) -eq $RUNNER_INDEX ]; then
    TESTS="$TESTS ${SPEC_FILES[$i]}"
  fi
done
```

**Why round-robin over timing-based?**
- CircleCI's timing-based splitting grouped similar tests together
- This caused race conditions with OAuth callback tests (Linear,
Shopify, Notion)
- Round-robin ensures even distribution and better test isolation
- Both CI systems now behave identically

### 2. **Frontend Test Optimization**
Enabled Vitest thread parallelization in `vite.config.ts`:

```typescript
pool: 'threads',
poolOptions: {
  threads: {
    singleThread: false,
  },
},
```

### 3. **CI Architecture Restructuring**
Split monolithic CI jobs into parallel stages:
- **Lint** (backend + frontend) - runs independently for fast feedback
- **Frontend tests** - runs in parallel with backend
- **Backend tests** - 16-way parallelized across runners
- **Coverage** - aggregates results from test jobs
- **Build** (CircleCI only) - final job for GitHub status check
compatibility

### 4. **Critical Test Optimization**
**report_builder_spec.rb**: Changed `before` to `before_all`
- Reduced execution time from **19 minutes to 1.2 minutes** (16x
speedup)
- Setup now runs once instead of 21 times
- Single biggest performance improvement after parallelization

---

## Test Stability Fixes (10 spec files)

Parallelization exposed latent test isolation issues that were fixed:

### Object Identity Comparisons (6 files)
Tests were comparing Ruby object instances instead of IDs:
- `spec/models/integrations/hook_spec.rb` - Use `.pluck(:id)` for
comparisons
- `spec/enterprise/models/captain/scenario_spec.rb` - Compare IDs
instead of objects
- `spec/models/notification_spec.rb` - Compare IDs for sort order
validation
- `spec/models/account_spec.rb` - Compare IDs in scope queries
- `spec/services/widget/token_service_spec.rb` - Compare class names
instead of class objects
- `spec/models/concerns/avatarable_shared.rb` - Use `respond_to` checks
for ActiveStorage

### Database Query Caching
- `spec/jobs/delete_object_job_spec.rb` - Added `.reload` to force fresh
database queries

### Test Expectations Timing
- `spec/jobs/mutex_application_job_spec.rb` - Removed flaky unlock
expectation after error block
  - Related to original PR #8770
- Expectation after error block never executes in parallel environments

### Timezone Handling
- `spec/mailers/account_notification_mailer_spec.rb` - Fixed date
parsing at timezone boundaries
  - Changed test time from 23:59:59Z to 12:00:00Z

### Test Setup
- `spec/builders/v2/report_builder_spec.rb` - Optimized with
`before_all`

---

##  CircleCI GitHub Integration Fix

### Problem
GitHub PR checks were stuck on "Waiting for status" even when all
CircleCI jobs passed. GitHub was expecting a job named `build` but the
workflow only had a workflow named "build".

### Solution
Added an explicit `build` job that runs after all other jobs:

```yaml
build:
  steps:
    - run:
        name: Legacy build aggregator
        command: echo "All main jobs passed"
  requires:
    - lint
    - coverage
```

This ensures GitHub's required status checks work correctly.


---

##  Testing & Validation

-  **GitHub Actions**: 9 successful runs, consistent 6m 26s - 7m 2s
runtime
-  **CircleCI**: 2 successful runs, consistent 7m 7s - 7m 21s runtime
-  Both CI systems produce identical, consistent results
-  GitHub PR status checks complete correctly
-  Success rate improved from 84% to 100% (recent runs)
-  No test regressions introduced
-  All flaky tests fixed (callback controllers, mutex jobs, etc.)

---

## 🎉 Summary

This PR delivers an **82% improvement** in test execution time across
both CI systems:

- **GitHub Actions tests**: 36m → 7m (81.5% faster)
- Backend tests specifically: 28m → 4m (86.6% faster)
- **CircleCI tests**: 42m → 7m (82.7% faster)
- **Developer feedback loop**: 5-6x faster
- **Test stability**: 84% → 100% success rate

### 📊 Total CI/CD Impact (All Optimizations)

Based on real November 2025 data, combining this PR with Docker build
optimization (PR #12859):

**Monthly Time Savings**: **1,358 hours/month** = **170
developer-days/month** = **7.7 FTE**

| System | Runs/Day | Before | After | Savings | Monthly Impact |
|--------|----------|---------|--------|---------|----------------|
| **GitHub Actions Tests** | 30.8 | 36.5m | 6.7m | 29.8m/run | 458 hrs
(34%) |
| **GitHub Actions Docker** | 25.2 | 34.2m | 5.0m | 29.2m/run | 369 hrs
(27%) |
| **CircleCI Tests** | 30.8 | 41.7m | 7.2m | 34.5m/run | 531 hrs (39%) |

*Data source: GitHub Actions Usage Metrics (November 2025, 18 days),
CircleCI Insights (30 days)*

The combined optimizations save the equivalent of **nearly 8 full-time
developers** worth of CI waiting time every month, significantly
improving developer velocity and reducing CI costs. All test isolation
issues exposed by parallelization have been fixed, ensuring reliable and
consistent results across both CI platforms.

woot woot !!!

---------
2025-11-19 15:32:48 +05:30
38af08534c fix: Captain response builder not getting triggered (#12729)
## Summary
- Fix captain response builder not getting triggered for cases where
responses are created as completed.

## Testing Instructions 
- Test articles with firecrawl
- Test articles without firecrawl
- Test PDF documents

---------

Co-authored-by: Pranav <pranav@chatwoot.com>
2025-10-28 18:31:08 -07:00
9fb0dfa4a7 feat: Add UI for custom tools (#12585)
### Tools list

<img width="2316" height="666" alt="CleanShot 2025-10-03 at 20 42 41@2x"
src="https://github.com/user-attachments/assets/ccbffd16-804d-4eb8-9c64-2d1cfd407e4e"
/>

### Tools form 

<img width="2294" height="2202" alt="CleanShot 2025-10-03 at 20 43
05@2x"
src="https://github.com/user-attachments/assets/9f49aa09-75a1-4585-a09d-837ca64139b8"
/>

## Response

<img width="800" height="2144" alt="CleanShot 2025-10-03 at 20 45 56@2x"
src="https://github.com/user-attachments/assets/b0c3c899-6050-4c51-baed-c8fbec5aae61"
/>

---------

Co-authored-by: Pranav <pranavrajs@gmail.com>
Co-authored-by: Pranav <pranav@chatwoot.com>
2025-10-06 09:05:54 -07:00
8bbb8ba5a4 feat(ee): Captain custom http tools (#12584)
To test this out, use the following PR:
https://github.com/chatwoot/chatwoot/pull/12585

---------

Co-authored-by: Pranav <pranavrajs@gmail.com>
2025-10-06 07:53:15 -07:00
Tanmay Deep SharmaandGitHub 1ba00075ce feat: Add BE changes for captain pdf support for faq generation (#12113) 2025-08-27 20:31:22 +05:30
b71a0da10d feat: scenario tools [CW-4597] (#11908)
Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Sojan Jose <sojan@pepalo.com>
2025-07-21 16:42:12 +05:30
Tanmay Deep SharmaGitHubMuhsin KelothCopilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>iamsivinSivin VargheseSojan Jose
61d10044a0 feat: Whatsapp embedded signup (#11612)
## Description

This PR introduces WhatsApp Embedded Signup functionality, enabling
users to connect their WhatsApp Business accounts through Meta's
streamlined OAuth flow without manual webhook configuration. This
significantly improves the user experience by automating the entire
setup process.

**Key Features:**

- Embedded signup flow using Facebook SDK and Meta's OAuth 2.0
- Automatic webhook registration and phone number configuration
- Enhanced provider selection UI with card-based design
- Real-time progress tracking during signup process
- Comprehensive error handling and user feedback


## Required Configuration

The following environment variables must be configured by administrators
before this feature can be used:
Super Admin Configuration (via
super_admin/app_config?config=whatsapp_embedded)

- `WHATSAPP_APP_ID`: The Facebook App ID for WhatsApp Business API
integration
- `WHATSAPP_CONFIGURATION_ID`: The Configuration ID for WhatsApp
Embedded Signup flow (obtained from Meta Developer Portal)
- `WHATSAPP_APP_SECRET`: The App Secret for WhatsApp Embedded Signup
flow (required for token exchange)
![Screenshot 2025-06-09 at 11 21
08 AM](https://github.com/user-attachments/assets/1615fb0d-27fc-4d9e-b193-9be7894ea93a)


## How Has This Been Tested?

#### Backend Tests (RSpec):

- Authentication validation for embedded signup endpoints
- Authorization code validation and error handling
- Missing business parameter validation
- Proper response format for configuration endpoint
- Unauthorized access prevention

#### Manual Test Cases:

- Complete embedded signup flow (happy path)
- Provider selection UI navigation
- Facebook authentication popup handling
- Error scenarios (cancelled auth, invalid business data, API failures)
- Configuration presence/absence behavior

## Related Screenshots:

![Screenshot 2025-06-09 at 7 48
18 PM](https://github.com/user-attachments/assets/34001425-df11-4d78-9424-334461e3178f)
![Screenshot 2025-06-09 at 7 48
22 PM](https://github.com/user-attachments/assets/c09f4964-3aba-4c39-9285-d1e8e37d0e33)
![Screenshot 2025-06-09 at 7 48
32 PM](https://github.com/user-attachments/assets/a34d5382-7a91-4e1c-906e-dc2d570c864a)
![Screenshot 2025-06-09 at 10 43
05 AM](https://github.com/user-attachments/assets/a15840d8-8223-4513-82e4-b08f23c95927)
![Screenshot 2025-06-09 at 10 42
56 AM](https://github.com/user-attachments/assets/8c345022-38b5-44c4-aba2-0cda81389c69)


Fixes
https://linear.app/chatwoot/issue/CW-2131/spec-for-whatsapp-cloud-channels-sign-in-with-facebook

---------

Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Co-authored-by: iamsivin <iamsivin@gmail.com>
Co-authored-by: Sivin Varghese <64252451+iamsivin@users.noreply.github.com>
Co-authored-by: Sojan Jose <sojan@pepalo.com>
2025-07-14 21:37:06 -07:00
93f18315cc feat: add Captain::Scenario Model and API [CW-4597] (#11907)
Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
2025-07-14 16:12:38 +05:30