Commit Graph
57 Commits
Author SHA1 Message Date
d20950c5b4 feat: scheduler fairness [AI-159] (#14425)
# Pull Request Template

## Description
Better scheduling and queueing mechanics for document auto-sync
- add jitter plan wise for document sync
- move auto-sync documents to purgeable queue

## Type of change

Please delete options that are not relevant.

- [x] New feature (non-breaking change which adds functionality)

## How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide
instructions so we can reproduce. Please also list any relevant details
for your test configuration.
locally tested and with specs

## Checklist:

- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] Any dependent changes have been merged and published in downstream
modules

---------

Co-authored-by: Sivin Varghese <64252451+iamsivin@users.noreply.github.com>
Co-authored-by: iamsivin <iamsivin@gmail.com>
Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
Co-authored-by: Sony Mathew <sony@chatwoot.com>
Co-authored-by: Vishnu Narayanan <iamwishnu@gmail.com>
2026-05-27 16:01:51 +05:30
Aakash BakhleandGitHub 37c8e7e699 fix: firecrawl long external link (#14566)
# Pull Request Template

## Description

Fixes urls going past 255 chars, this is because of arabic urls, where
each character balloons to 8-9 characters and goes past the 255 limit

## Type of change

Please delete options that are not relevant.

- [x] Bug fix (non-breaking change which fixes an issue)

## How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide
instructions so we can reproduce. Please also list any relevant details
for your test configuration.
specs


## Checklist:

- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] Any dependent changes have been merged and published in downstream
modules
2026-05-26 14:07:07 +05:30
Shivam MishraandGitHub 3d20a7b049 feat: generate Help Center for Onboarding (#14370)
## Manually triggering help center generation

Open a Rails console (`bundle exec rails console`):

```ruby
account = Account.find(<ACCOUNT_ID>)
user    = account.users.first

# Optional: refresh brand info from the customer's website
domain = 'example.com'
result = WebsiteBrandingService.new("noreply@#{domain}").perform
account.update!(
  name: result[:title].presence || account.name,
  custom_attributes: account.custom_attributes.merge('website' => domain, 'brand_info' => result)
)

# Optional: wipe existing portals so a fresh one is created
account.portals.destroy_all

Onboarding::HelpCenterCreationService.new(account, user).perform
```

Sidekiq must be running — articles are written by
`Onboarding::HelpCenterArticleGenerationJob`. Avoid running on
production; generation calls the LLM provider.


### Generation flow (Happy Path) 

```mermaid
sequenceDiagram
    autonumber

    participant Kickoff as HelpCenterCreationService
    participant DB as DB
    participant GenJob as HelpCenterArticleGenerationJob
    participant Curator as HelpCenterCurator
    participant Firecrawl as Firecrawl
    participant CuratorLLM as Curation LLM
    participant Redis as Redis Progress
    participant WriterJob as HelpCenterArticleWriterJob
    participant Builder as HelpCenterArticleBuilder
    participant WriterLLM as Writer LLM
    participant Cable as ActionCable

    Kickoff->>DB: Create portal for account<br/>homepage_link=https://chatwoot.com
    Kickoff->>DB: Attach brand logo if available
    Kickoff->>GenJob: Enqueue generation job<br/>account_id, portal_id, user_id, generation_id

    GenJob->>Curator: Curate help center plan
    Curator->>Firecrawl: map https://chatwoot.com<br/>search: docs help support faq
    Firecrawl-->>Curator: Return discovered links
    Curator->>CuratorLLM: Select categories + article plans<br/>from discovered links only
    CuratorLLM-->>Curator: Return categories, articles, allowed_urls

    GenJob->>DB: Create portal categories
    GenJob->>GenJob: Stamp articles with category_id
    GenJob->>GenJob: Filter article URLs against allowed_urls
    GenJob->>GenJob: Drop articles with no category<br/>or no approved source URLs

    GenJob->>Redis: Start progress<br/>status=generating, total=N, finished=0

    loop For each approved article
      GenJob->>WriterJob: Enqueue writer job<br/>title, category_id, approved URLs
    end

    par Writer jobs run independently
      WriterJob->>Builder: Build article from approved URLs
      Builder->>Firecrawl: batch_scrape approved URLs
      Firecrawl-->>Builder: Return Markdown source pages
      Builder->>WriterLLM: Rewrite sources into one article
      WriterLLM-->>Builder: Return title, description, Markdown content
      Builder->>DB: Create draft portal article<br/>meta.source_urls
      WriterJob->>Redis: Increment finished count
      WriterJob->>Cable: Broadcast help_center.article_generated
    end

    WriterJob->>Redis: If finished >= total<br/>mark status=completed
    WriterJob->>Cable: Broadcast help_center.generation_completed
```

### Redis State Management

```mermaid
 stateDiagram-v2
    [*] --> active_pointer_set
    active_pointer_set --> generating: generation job creates valid plan
    active_pointer_set --> skipped: curation skipped/failed

    generating --> generating: each writer job increments finished
    generating --> completed: finished == total
    generating --> ignored_completion: generation_id superseded

    skipped --> [*]
    completed --> [*]
    ignored_completion --> [*]
```
2026-05-21 16:25:01 +05:30
Aakash BakhleandGitHub d7d1e4113c fix: captain auto sync scheduler resilience (#14379)
# Pull Request Template

## Description

skip documents that fail with ActiveRecord errors possibly due to
stale/corrupt data and not crash scheduler

How did we find out about this error?

before October 28th, 2025, we did not have url normalisation.
so we had document rows as: 
id: 123 `https://example.com` status: `in_progress` --> likely stuck
crawl
id 234: `https://example.com/` status: `available` 

When the schedule sync job ran, it ran an `document.update!(sync_status:
:syncing, last_sync_attempted_at: Time.current)` on the 234 one since it
was `available`

now `update!` runs `before_validation :normalize_external_link`
so `https://example.com/` became `https://example.com`

which invalidated:
`validates :external_link, uniqueness: { scope: :assistant_id }`

so the scheduler crashed.

This PR logs the skipped ones with their errors and continues to pick
other documents to scheduler doesn't crash

## Type of change

- [x] Bug fix (non-breaking change which fixes an issue)

## How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide
instructions so we can reproduce. Please also list any relevant details
for your test configuration.
spec

## Checklist:

- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] Any dependent changes have been merged and published in downstream
modules
2026-05-06 18:07:22 +05:30
Aakash BakhleandGitHub 70f799ab35 fix(captain): add v1 handoff classifier [AI-137] (#14337)
# Pull Request Template

## Description

Captain (v1) makes false promises by saying it will handoff but doesn't.
This happens due to an exact string match comparison and the prompt
gives the model a lot of responsibilities:
- identity
- what to respond
- obey custom instructions
- decide on tool calls

This PR decouples responsibility, the core prompt responds, and an
additional llm call evaluates if handoff was needed or not after that
message.

## Type of change

- [x] Bug fix (non-breaking change which fixes an issue)

## How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide
instructions so we can reproduce. Please also list any relevant details
for your test configuration.

Locally


## Checklist:

- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] Any dependent changes have been merged and published in downstream
modules
2026-05-05 14:34:31 +05:30
Aakash BakhleandGitHub d00867d636 fix: captain auto sync scheduler config (#14336) 2026-05-04 13:37:25 +05:30
Aakash BakhleandGitHub c09206c22e fix: flaky time based document sync spec (#14320)
fixes:
https://app.circleci.com/pipelines/github/chatwoot/chatwoot/111572/workflows/210e3618-301a-4a8f-a411-826214965441/jobs/146263
2026-04-29 17:36:14 +05:30
Aakash BakhleandGitHub 568aae875b feat: wire up auto-sync job backend [AI-150] (#14117)
# Pull Request Template

## Description

- Wires up Controllers to auto-sync job
- adds plan based sync schedule
- a scheduler that runs every hour to check syncable documents
- guards the whole feature behind feature flag by reclaiming
`twilio_content_templates`
- Adds a global and account level cap on how many documents to enqueue
to prevent sudden burst at first run
- some refactor to simplify code
- specs

Fixes # (issue)

## Type of change

- [x] New feature (non-breaking change which adds functionality)

## How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide
instructions so we can reproduce. Please also list any relevant details
for your test configuration.

specs and locally

## Checklist:

- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] Any dependent changes have been merged and published in downstream
modules
2026-04-29 14:47:14 +05:30
Aakash BakhleandGitHub a651949c33 fix: improve FAQ generation [AI-145] (#14062)
# Pull Request Template

## Description

- Fetch main content only from Firecrawl, exclude some tags to remove
boilerplate
- Prompt changes for FAQ generation

## Type of change

Please delete options that are not relevant.

- [x] Bug fix (non-breaking change which fixes an issue)

## How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide
instructions so we can reproduce. Please also list any relevant details
for your test configuration.

tested locally

## Checklist:

- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] Any dependent changes have been merged and published in downstream
modules
2026-04-27 13:01:44 +05:30
PranavandGitHub 751c28d94d feat(ee): Add article translation via LLM in help center (#14136)
Adds the ability to translate help center articles to other languages using Captain's LLM infrastructure. Translated articles are created as drafts linked to the source article.

Fixes
https://linear.app/chatwoot/issue/CW-6901/translate-article-to-another-language

**How to test**

1. Navigate to Help Center → Articles for a portal with multiple locales
2. Click the three-dot menu on any article → "Translate"
3. Select a target language and category → click Translate
4. Switch to the target locale — the translated article appears as a
draft
5. Try translating the same article again — a warning shows the existing
translation with a link to open it in a new tab
6. Click "Overwrite and translate" to replace the existing translation


https://github.com/user-attachments/assets/1d2e991b-f0ac-403a-bcc1-2181b5731ea4
2026-04-24 08:51:26 -07:00
00837019b5 fix(captain): display handoff message to customer in V2 flow (#13885)
HandoffTool changes conversation status but only posts a private note.
ResponseBuilderJob now detects the tool flag and creates the public
handoff message that was previously only shown in V1.

# Pull Request Template

## Description

Captain V2 was silently forwarding conversations to humans without
showing a handoff message to the customer. The conversation appeared to
just stop
responding.

Root cause: In V2, HandoffTool calls bot_handoff! during agent
execution, which changes conversation status from pending to open. By
the time control returns
to ResponseBuilderJob#process_response, the conversation_pending? guard
returns early - skipping create_handoff_message entirely. The V1 flow
didn't have this
problem because AssistantChatService just returns a string token
(conversation_handoff) and lets ResponseBuilderJob handle everything.

What changed:

1. AgentRunnerService now surfaces the handoff_tool_called flag (already
tracked internally for usage metadata) in its response hash.
2. ResponseBuilderJob#handoff_requested? detects handoffs from both V1
(response token) and V2 (tool flag).
3. ResponseBuilderJob#process_response checks handoff_requested? before
the conversation_pending? guard, so V2 handoffs are processed even when
the status has
already changed.
4. ResponseBuilderJob#process_action('handoff') captures
conversation_pending? before calling bot_handoff! and uses that snapshot
to guard both bot_handoff!
and the OOO message - preventing double-execution when V2's HandoffTool
already ran them.

New V2 handoff flow:
AgentRunnerService
  → agent calls HandoffTool (creates private note, calls bot_handoff!)
  → returns response with handoff_tool_called: true

ResponseBuilderJob#process_response
  → handoff_requested? detects the flag
  → process_action('handoff')
    → create_handoff_message (public message for customer)
    → bot_handoff! skipped (conversation_pending? is false)
    → OOO skipped (conversation_pending? is false)

Fixes #13881

## Type of change

Please delete options that are not relevant.

- [x] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality not to work as expected)
- [ ] This change requires a documentation update

## How Has This Been Tested?

- Update existing response_builder_job_spec.rb covering the V2 handoff
path, V2 normal response path, and V1 regression
- Updated existing agent_runner_service_spec.rb expectations for the new
handoff_tool_called key and added a context for when the flag is true

## Checklist:

- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [ ] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] Any dependent changes have been merged and published in downstream
modules

---------

Co-authored-by: Aakash Bakhle <48802744+aakashb95@users.noreply.github.com>
Co-authored-by: aakashb95 <aakashbakhle@gmail.com>
2026-04-08 17:30:07 +05:30
Aakash BakhleandGitHub 290dd3abf5 feat: allow captain to access contact attributes (#13850)
# Pull Request Template

## Description

Captain v1 does not have access to contact attributes. Added a toggle to
let user choose if they want contact information available to Captain.

## Type of change

- [x] Bug fix (non-breaking change which fixes an issue)

## How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide
instructions so we can reproduce. Please also list any relevant details
for your test configuration.

Specs and locally
<img width="1924" height="740" alt="CleanShot 2026-03-19 at 18 48 19@2x"
src="https://github.com/user-attachments/assets/353cfeaa-cd58-40eb-89e7-d660a1dc1185"
/>

![Uploading CleanShot 2026-03-19 at 18.53.26@2x.png…]()


## Checklist:

- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] Any dependent changes have been merged and published in downstream
modules
2026-03-20 16:15:06 +05:30
Aakash BakhleandGitHub a90ffe6264 feat: Add force legacy auto-resolve flag (#13804)
# Pull Request Template

## Description

Add account setting and store_accessor for
`captain_force_legacy_auto_resolve`.
Enterprise job now skips LLM evaluation when this flag is true and falls
back to legacy time-based resolution. Add spec to cover the fallback.


## Type of change

We recently rolled out Captain deciding if a conversation is resolved or
not. While it is an improvement for majority of customers, some still
prefer the old way of auto-resolving based on inactivity. This PR adds a
check.

## How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide
instructions so we can reproduce. Please also list any relevant details
for your test configuration.

legacy_auto_resolve = true

<img width="1282" height="848" alt="CleanShot 2026-03-13 at 19 55 55@2x"
src="https://github.com/user-attachments/assets/dfdcc5d5-6d21-462b-87a6-a5e1b1290a8b"
/>


legacy_auto_resolve = false
<img width="1268" height="864" alt="CleanShot 2026-03-13 at 20 00 50@2x"
src="https://github.com/user-attachments/assets/f4719ec6-922a-4c3b-bc45-7b29eaced565"
/>



## Checklist:

- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] Any dependent changes have been merged and published in downstream
modules
2026-03-13 15:04:58 -07:00
d6d38cdd7d feat: captain decides if conversation should be resolved or kept open (#13336)
# Pull Request Template

## Description

captain decides if conversation should be resolved or open

Fixes
https://linear.app/chatwoot/issue/AI-91/make-captain-resolution-time-configurable

Update: Added 2 entries in reporting events:
`conversation_captain_handoff` and `conversation_captain_resolved`

## Type of change

Please delete options that are not relevant.

- [x] New feature (non-breaking change which adds functionality)
- [x] This change requires a documentation update

## How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide
instructions so we can reproduce. Please also list any relevant details
for your test configuration.

LLM call decides that conversation is resolved, drops a private note
<img width="1228" height="438" alt="image"
src="https://github.com/user-attachments/assets/fb2cf1e9-4b2b-458b-a1e2-45c53d6a0158"
/>

LLM call decides conversation is still open as query was not resolved
<img width="1215" height="573" alt="image"
src="https://github.com/user-attachments/assets/2d1d5322-f567-487e-954e-11ab0798d11c"
/>


## Checklist:

- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented on my code, particularly in hard-to-understand
areas
- [] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] Any dependent changes have been merged and published in downstream
modules

---------

Co-authored-by: Sojan Jose <sojan@pepalo.com>
2026-03-13 10:03:58 +05:30
059506b1db feat: Add automatic favicon fetching for companies (#13013)
## Summary

This Enterprise-only feature automatically fetches a favicon for
companies created with a domain, and adds a batch task to backfill
missing avatars for existing companies. The flow only targets companies
that do not already have an attached avatar, so existing avatars are
left untouched.


## Demo 



https://github.com/user-attachments/assets/d050334e-769f-4e46-b6e7-f7423727a192



## What changed

- Added `Avatar::AvatarFromFaviconJob` to build a Google favicon URL
from the company domain and fetch it through `Avatar::AvatarFromUrlJob`
- Triggered favicon fetching from `Company` with `after_create_commit`
- Added `Companies::FetchAvatarsJob` to batch existing companies that
are missing avatars
- Added `companies:fetch_missing_avatars` under `enterprise/lib/tasks`
- Kept the company-specific implementation inside the Enterprise
boundary
- Stubbed the new favicon request in unrelated specs that now hit this
callback indirectly
- Updated a couple of CI-sensitive specs that were failing due to
callback side effects / reload-safe exception assertions

## How to verify

1. Create a company in Enterprise with a valid domain and no avatar.
2. Confirm that a favicon-based avatar gets attached shortly after
creation.
3. Create another company with a domain and an avatar already attached.
4. Confirm that the existing avatar is not replaced.
5. Run `companies:fetch_missing_avatars`.
6. Confirm that existing companies without avatars get one, while
companies that already have avatars remain unchanged.

## Notes

- This change does not refresh or overwrite existing company avatars
- Favicon fetching only runs for companies with a present domain
- The branch includes the latest `develop`

---------

Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
Co-authored-by: Sojan Jose <sojan@pepalo.com>
2026-03-05 18:51:28 -08:00
Aakash BakhleandGitHub 374d2258c7 fix: captain talking over support agent (#13673) 2026-03-03 16:13:34 +05:30
c08fa631a9 feat: Add temporary account setting to disable Captain auto-resolve (#13680)
Add a temporary `captain_disable_auto_resolve` boolean setting on
accounts to prevent Captain from resolving conversations. Guards both
the scheduled resolution job and the assistant's resolve tool.

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-27 09:37:00 -08:00
df92fd12cb fix: bot handoff should set waiting time (#13417)
Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
2026-02-27 15:31:49 +05:30
Aakash BakhleandGitHub dcc72ee63c fix: consistent instrumentation with conversation.display_id (#13194) 2026-01-08 12:27:44 +05:30
3fce56c98f fix: captain template message conflict (#13048)
Co-authored-by: aakashb95 <aakash@chatwoot.com>
Co-authored-by: Shivam Mishra <scm.mymail@gmail.com>
Co-authored-by: Vishnu Narayanan <iamwishnu@gmail.com>
2025-12-15 15:47:26 +05:30
Aakash BakhleandGitHub 1de8d3e56d feat: legacy features to ruby llm (#12994) 2025-12-11 14:17:28 +05:30
1ef945de7b feat: Instrument captain (#12949)
Co-authored-by: aakashb95 <aakash@chatwoot.com>
2025-11-28 15:12:55 +05:30
1fbdd68222 feat: Add company auto-association for contacts (CW-5726 Part 2) (#12711)
## Description

Implements real-time company auto-association for contacts based on
email domains. This is **Part 2** of the company model production
rollout (CW-5726).

**Task:**
- When a contact is created with a business email, automatically create
and associate a company from the email domain
- When a contact is updated with an email for the first time (email was
previously nil), associate with a company
- Preserve existing company associations when email changes to avoid
user confusion
- Skip free email providers and disposable domains

**Dependencies:**
⚠️ Requires PR #12657 (Part 1: Backfill migration) to be merged first

**Linear ticket:**
[CW-5726](https://linear.app/chatwoot/issue/CW-5726/company-model-setting-it-up-on-production)

## Type of change

- [x] New feature (non-breaking change which adds functionality)

## How Has This Been Tested?

- Service specs: Tests business email detection, company creation,
association logic, edge cases (existing companies, free emails, nil
emails)
- Integration specs: Tests full callback flow for contact create/update
scenarios
- All tests passing: 10 examples, 0 failures
- RuboCop: 0 offenses

## Checklist:

- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented on my code, particularly in hard-to-understand
areas
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged and published in downstream
modules (PR #12657 pending)

---------

Co-authored-by: Sojan Jose <sojan@pepalo.com>
2025-11-03 20:36:13 +05:30
ef54f07d5b feat: Add company backfill migration for existing contacts (Part 1) (#12657)
## Description

Implements company backfill migration infrastructure for existing
contacts. This is **Part 1 of 2** for the company model production
rollout as described in
[CW-5726](https://linear.app/chatwoot/issue/CW-5726/company-model-setting-it-up-on-production).

Creates jobs and services to associate existing contacts with companies
based on their email domains, filtering out free email providers (gmail,
yahoo, etc.) and disposable addresses.
 

**What's included:**
- Business email detector service with ValidEmail2 (uses
`disposable_domain?` to avoid DNS lookups)
- Per-account batch job to process contacts for one account
- Orchestrator job to iterate all accounts
- Rake task: `bundle exec rake companies:backfill`

~~*NOTE*: I'm using a hard-coded approach to determine if something is a
"business" email by filtering out emails that are usually personal. I've
also added domains that are common to some of our customers' regions.
This should be simpler. I looked into `Valid_Email2` and I couldn't find
anything to dictate whether an email is a personal email or a business
one. I don't think the approach used in the frontend is valid here.~~
UPDATE: Using `email_provider_info` gem instead.


**Pending - Part 2 (separate PR):** Real-time company creation for new
contacts

## Type of change

- [x] New feature (non-breaking change which adds functionality)

## How Has This Been Tested?

```bash
# Run all new tests
bundle exec rspec spec/enterprise/services/companies/business_email_detector_service_spec.rb \\
                   spec/enterprise/jobs/migration/company_account_batch_job_spec.rb \\
                   spec/enterprise/jobs/migration/company_backfill_job_spec.rb

# Run RuboCop
bundle exec rubocop enterprise/app/services/companies/business_email_detector_service.rb \\
                     enterprise/app/jobs/migration/company_account_batch_job.rb \\
                     enterprise/app/jobs/migration/company_backfill_job.rb \\
                     lib/tasks/companies.rake
```

**Performance optimization:**
- Uses `disposable_domain?` instead of `disposable?` to avoid DNS MX
lookups (discovered via tcpdump analysis - `disposable?` was making
network calls for every email, causing 100x slowdown)

## Checklist:

- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged and published in downstream
modules

---------

Co-authored-by: Sojan Jose <sojan@pepalo.com>
2025-11-03 20:03:47 +05:30
38af08534c fix: Captain response builder not getting triggered (#12729)
## Summary
- Fix captain response builder not getting triggered for cases where
responses are created as completed.

## Testing Instructions 
- Test articles with firecrawl
- Test articles without firecrawl
- Test PDF documents

---------

Co-authored-by: Pranav <pranav@chatwoot.com>
2025-10-28 18:31:08 -07:00
Sojan JoseandGitHub eabdfc8168 chore(sidekiq): log ActiveJob class and job_id on dequeue (#12704)
## Context

Sidekiq logs only showed the Sidekiq wrapper class and JID, which wasn’t
helpful when debugging ActiveJobs.

## Changes

- Updated `ChatwootDequeuedLogger` to log the actual `ActiveJob class`
and `job_id` instead of the generic Sidekiq wrapper and JID.

> Example
> ```
> Dequeued ActionMailer::MailDeliveryJob
123e4567-e89b-12d3-a456-426614174000 from default
> ```

- Remove sidekiq worker and unify everything to `ActiveJob`
2025-10-22 20:20:37 -07:00
Shivam MishraandGitHub 7a453f50f4 feat: update users on SAML setup and destroy [CW-2958][CW-5612] (#12346) 2025-09-15 21:20:22 +05:30
Tanmay Deep SharmaandGitHub 1ba00075ce feat: Add BE changes for captain pdf support for faq generation (#12113) 2025-08-27 20:31:22 +05:30
PranavandGitHub 7f56cd92f8 chore: Update the prompts to include language of the account for FAQs (#12280)
There were customer reported issues with FAQs which were generated in a
different langauge than what they were expecting. The reason behind this
was that the language of the account was not considered in the prompt
provided. If the language of the content was say Spanish, and the
account locale was english. The output was not predicable. The output
depends on the model and the execution time.

This PR would update the prompt to behave consistently with the account
locale. Even though the content provided is in a different language, it
would generate FAQs in the account locale.

Changes:
- Updated the prompt to include a detailed expectation of the FAQs
quality along with the language
- Added specs for the services where the prompt generator is called.

Tested the prompt using Phoenix playground across GPT 5, GPT 4.1, GPT
4.0. The reasoning setting for GPT 5 needs to be low so that it doesn't
generate random questions like "What was this updated?"
2025-08-22 10:03:52 -07:00
c6be04cdc1 feat: scenario agents & runner (#11944)
Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Sojan Jose <sojan@pepalo.com>
Co-authored-by: Pranav <pranav@chatwoot.com>
Co-authored-by: Sivin Varghese <64252451+iamsivin@users.noreply.github.com>
2025-08-14 12:39:21 +05:30
PranavandGitHub 51b9fd8eca fix: Disable IMAP inboxes that requires authorization (#12092)
This PR disables queueing IMAP sync jobs for emails channels that 
- are in free plan if on Chatwoot cloud.
- requires authorization
2025-08-01 16:32:29 -07:00
420be64c45 chore: Automate SSL with Cloudflare (#12021)
This PR adds support for automatic SSL issuance using Cloudflare when a
custom domain is updated.

- Introduced a cloudflare configuration. If present, the system will
attempt to issue an SSL certificate via Cloudflare whenever a custom
domain is added or changed.
- SSL verification is handled using an HTTP challenge.
- The job will store the HTTP challenge response provided by Cloudflare
and serve it under the /.well-known/cf path automatically.

How to test:

- Create a Cloudflare zone for your domain and copy the Zone ID.
- Generate a Cloudflare API token with the required SSL certificate
permissions.
- Set the Fallback Origin under SSL -> Custom HostName to the Chatwoot
installation.
- Add or update a custom domain and verify that the SSL certificate is
automatically issued.

---------

Co-authored-by: Sojan Jose <sojan@pepalo.com>
2025-07-24 13:09:06 +04:00
Tanmay Deep SharmaandGitHub 5b9f997fa0 feat: Add the support for images in Captain (#11850) 2025-07-11 23:28:46 +07:00
Muhsin KelothandGitHub a657b45bd1 feat(revert): "feat: captain image support" (#11841)
Reverts chatwoot/chatwoot#11730
2025-07-01 00:12:07 +05:30
811eb66615 feat: Add support for image files in Captain (#11730)
# Pull Request Template

## Linear links:
-
https://linear.app/chatwoot/issue/CW-4479/if-image-is-sent-by-the-customer-send-it-to-openai

## Description

This pull request adds “Captain image support” to Chatwoot. It
introduces multimodal message handling so that when a customer sends an
image, Captain can forward the file to OpenAI’s vision endpoint,
generate a caption/analysis
## Type of change

Please delete options that are not relevant.

- [x] New feature (non-breaking change which adds functionality)

## How Has This Been Tested?

<img width="891" alt="image"
src="https://github.com/user-attachments/assets/c7cc98ed-cc44-4865-a53a-83d129e2fe2c"
/>



## Checklist:

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged and published in downstream
modules

---------

Co-authored-by: Pranav <pranav@chatwoot.com>
2025-06-25 19:16:09 -07:00
8bc00f707b feat(ee): Add transcription support for audio messages (#11670)
<img width="419" alt="Screenshot 2025-06-03 at 4 25 37 PM"
src="https://github.com/user-attachments/assets/4b6ddd11-9b91-4981-a571-83746cc4d40b"
/>


Fixes https://github.com/chatwoot/chatwoot/issues/10182

---------

Co-authored-by: Sojan Jose <sojan@pepalo.com>
2025-06-05 18:29:37 -05:00
PranavandGitHub 3a0b5f387d fix: Update specs, add background response job implementation for copilot threads (#11600)
- Enable jobs by default when a copilot thread or a message is created.
- Rename thread_id to copilot_thread_id to keep it consistent with the
model name
- Add a spec for search_linear_issues service
2025-05-27 14:10:27 -06:00
eba24ce275 chore: Add activity message for conversation resolutions by Captain (#11492)
Currently, when Captain resolves a conversation, the same activity
message used for auto-resolution is shown. This has caused confusion for
users.
With this change, a distinct activity message will be displayed
specifically for resolutions performed by Captain, improving clarity.


Fixes
https://linear.app/chatwoot/issue/CW-4289/incorrect-activity-message-for-conversations-resolved-by-captain-auto#comment-d2991763

Co-authored-by: Pranav <pranav@chatwoot.com>
2025-05-16 11:57:07 -07:00
PranavandGitHub fb6409508b feat: Allow customizing the responses, flows in Captain (#11385)
- Ability to provide custom instructions to captain

<img width="1107" alt="Screenshot 2025-04-28 at 6 11 43 PM"
src="https://github.com/user-attachments/assets/f94cbccc-b4d8-48fd-b6b9-55524129bc50"
/>
2025-04-29 15:42:15 -07:00
PranavandGitHub c7d259d5fd chore: Update the behavior of Captain resolutions (#10794)
This PR ensures that only conversations from quick conversation channels
are resolved, avoiding resolutions on the email channel (we still need
to improve the UX here). It also updates the FAQ generation logic,
limiting it to conversations that had at least one human interaction.
2025-02-03 16:25:08 +05:30
PranavandGitHub cf0975ad94 fix: Use current_available instead available to compute the document limit (#10776)
The available limit for documents is now stored in
captain.documents.current_available. This PR fixes the limit sent on
crawler job.
2025-01-28 10:55:46 +05:30
be8205657e fix: Added authentication to FireCrawl API, remove unused RobinAI references (#10737)
- Fixed Firecrawl webhook payloads to ensure proper data handling and
delivery.
- Removed unused Robin AI code to improve codebase cleanliness and
maintainability.
- Implement authentication for the Firecrawl endpoint to improve
security. A key is generated to secure the webhook URLs from FireCrawl.

---------

Co-authored-by: Pranav <pranavrajs@gmail.com>
2025-01-22 18:14:25 -08:00
Shivam MishraandGitHub 3b366f43e6 feat: setup captain limits (#10713)
This pull request introduces several changes to implement and manage
usage limits for the Captain AI service. The key changes include adding
configuration for plan limits, updating error messages, modifying
controllers and models to handle usage limits, and updating tests to
ensure the new functionality works correctly.

## Implementation Checklist

- [x] Ability to configure captain limits per check
- [x] Update response for `usage_limits` to include captain limits
- [x] Methods to increment or reset captain responses limits in the
`limits` column for the `Account` model
- [x] Check documents limit using a count query
- [x] Ensure Captain hand-off if a limit is reached
- [x] Ensure limits are enforced for Copilot Chat
- [x] Ensure limits are reset when stripe webhook comes in 
- [x] Increment usage for FAQ generation and Contact notes
- [x] Ensure documents limit is enforced

These changes ensure that the Captain AI service operates within the defined usage limits for different subscription plans, providing appropriate error messages and handling when limits are exceeded.
2025-01-23 01:23:18 +05:30
0b4028b95d feat: Add support for the references in FAQs (#10699)
Currently, it’s unclear whether an FAQ item is generated from a
document, derived from a conversation, or added manually.

This PR resolves the issue by providing visibility into the source of
each FAQ. Users can now see whether an FAQ was generated or manually
added and, if applicable, by whom.

- Move the document_id to a polymorphic relation (documentable).
- Updated the APIs to accommodate the change.
- Update the service to add corresponding references. 
- Updated the specs.

<img width="1007" alt="Screenshot 2025-01-15 at 11 27 56 PM"
src="https://github.com/user-attachments/assets/7d58f798-19c0-4407-b3e2-748a919d14af"
/>

---------

Co-authored-by: Sivin Varghese <64252451+iamsivin@users.noreply.github.com>
2025-01-16 15:27:30 +05:30
d070743383 feat(ee): Add Captain features (#10665)
Migration Guide: https://chwt.app/v4/migration

This PR imports all the work related to Captain into the EE codebase. Captain represents the AI-based features in Chatwoot and includes the following key components:

- Assistant: An assistant has a persona, the product it would be trained on. At the moment, the data at which it is trained is from websites. Future integrations on Notion documents, PDF etc. This PR enables connecting an assistant to an inbox. The assistant would run the conversation every time before transferring it to an agent.
- Copilot for Agents: When an agent is supporting a customer, we will be able to offer additional help to lookup some data or fetch information from integrations etc via copilot.
- Conversation FAQ generator: When a conversation is resolved, the Captain integration would identify questions which were not in the knowledge base.
- CRM memory: Learns from the conversations and identifies important information about the contact.

---------

Co-authored-by: Vishnu Narayanan <vishnu@chatwoot.com>
Co-authored-by: Sojan <sojan@pepalo.com>
Co-authored-by: iamsivin <iamsivin@gmail.com>
Co-authored-by: Sivin Varghese <64252451+iamsivin@users.noreply.github.com>
2025-01-14 16:15:47 -08:00
8d8ec23aa1 chore: [Snyk] Security upgrade rspec-rails from 6.1.4 to 6.1.5 (#10066)
![snyk-top-banner](https://github.com/andygongea/OWASP-Benchmark/assets/818805/c518c423-16fe-447e-b67f-ad5a49b5d123)

### Snyk has created this PR to fix 1 vulnerabilities in the rubygems
dependencies of this project.

#### Snyk changed the following file(s):

- `Gemfile`



<details>
<summary>⚠️ <b>Warning</b></summary>

```
Failed to update the Gemfile.lock, please update manually before merging.
```

</details>



#### Vulnerabilities that will be fixed with an upgrade:

|  | Issue | Score | 

:-------------------------:|:-------------------------|:-------------------------
![medium
severity](https://res.cloudinary.com/snyk/image/upload/w_20,h_20/v1561977819/icon/m.png
'medium severity') | Web Cache Poisoning
<br/>[SNYK-RUBY-RACK-1061917](https://snyk.io/vuln/SNYK-RUBY-RACK-1061917)
| &nbsp;&nbsp;**616**&nbsp;&nbsp;




---

> [!IMPORTANT]
>
> - Check the changes in this PR to ensure they won't cause issues with
your project.
> - Max score is 1000. Note that the real score may have changed since
the PR was raised.
> - This PR was automatically created by Snyk using the credentials of a
real user.

---

**Note:** _You are seeing this because you or someone else with access
to this repository has authorized Snyk to open fix PRs._

For more information: <img
src="https://api.segment.io/v1/pixel/track?data=eyJ3cml0ZUtleSI6InJyWmxZcEdHY2RyTHZsb0lYd0dUcVg4WkFRTnNCOUEwIiwiYW5vbnltb3VzSWQiOiJhMWE2MzkzZS03ODdhLTRmYWItOGY1MS0zZjdmN2YzNzVlZDYiLCJldmVudCI6IlBSIHZpZXdlZCIsInByb3BlcnRpZXMiOnsicHJJZCI6ImExYTYzOTNlLTc4N2EtNGZhYi04ZjUxLTNmN2Y3ZjM3NWVkNiJ9fQ=="
width="0" height="0"/>
🧐 [View latest project
report](https://app.snyk.io/org/chatwoot/project/b7197bbd-6200-4f23-931d-c39928584360?utm_source&#x3D;github&amp;utm_medium&#x3D;referral&amp;page&#x3D;fix-pr)
📜 [Customise PR
templates](https://docs.snyk.io/scan-using-snyk/pull-requests/snyk-fix-pull-or-merge-requests/customize-pr-templates)
🛠 [Adjust project
settings](https://app.snyk.io/org/chatwoot/project/b7197bbd-6200-4f23-931d-c39928584360?utm_source&#x3D;github&amp;utm_medium&#x3D;referral&amp;page&#x3D;fix-pr/settings)
📚 [Read about Snyk's upgrade
logic](https://support.snyk.io/hc/en-us/articles/360003891078-Snyk-patches-to-fix-vulnerabilities)

---

**Learn how to fix vulnerabilities with free interactive lessons:**

🦉 [Learn about vulnerability in an interactive lesson of Snyk
Learn.](https://learn.snyk.io/?loc&#x3D;fix-pr)

[//]: #
'snyk:metadata:{"customTemplate":{"variablesUsed":[],"fieldsUsed":[]},"dependencies":[{"name":"rspec-rails","from":"6.1.4","to":"6.1.5"}],"env":"prod","issuesToFix":[{"exploit_maturity":"Proof
of
Concept","id":"SNYK-RUBY-RACK-1061917","priority_score":616,"priority_score_factors":[{"type":"exploit","label":"Proof
of
Concept","score":107},{"type":"fixability","label":true,"score":214},{"type":"cvssScore","label":"5.9","score":295},{"type":"scoreVersion","label":"v1","score":1}],"severity":"medium","title":"Web
Cache Poisoning"},{"exploit_maturity":"Proof of
Concept","id":"SNYK-RUBY-RACK-1061917","priority_score":616,"priority_score_factors":[{"type":"exploit","label":"Proof
of
Concept","score":107},{"type":"fixability","label":true,"score":214},{"type":"cvssScore","label":"5.9","score":295},{"type":"scoreVersion","label":"v1","score":1}],"severity":"medium","title":"Web
Cache Poisoning"},{"exploit_maturity":"Proof of
Concept","id":"SNYK-RUBY-RACK-1061917","priority_score":616,"priority_score_factors":[{"type":"exploit","label":"Proof
of
Concept","score":107},{"type":"fixability","label":true,"score":214},{"type":"cvssScore","label":"5.9","score":295},{"type":"scoreVersion","label":"v1","score":1}],"severity":"medium","title":"Web
Cache Poisoning"},{"exploit_maturity":"Proof of
Concept","id":"SNYK-RUBY-RACK-1061917","priority_score":616,"priority_score_factors":[{"type":"exploit","label":"Proof
of
Concept","score":107},{"type":"fixability","label":true,"score":214},{"type":"cvssScore","label":"5.9","score":295},{"type":"scoreVersion","label":"v1","score":1}],"severity":"medium","title":"Web
Cache
Poisoning"}],"prId":"a1a6393e-787a-4fab-8f51-3f7f7f375ed6","prPublicId":"a1a6393e-787a-4fab-8f51-3f7f7f375ed6","packageManager":"rubygems","priorityScoreList":[616],"projectPublicId":"b7197bbd-6200-4f23-931d-c39928584360","projectUrl":"https://app.snyk.io/org/chatwoot/project/b7197bbd-6200-4f23-931d-c39928584360?utm_source=github&utm_medium=referral&page=fix-pr","prType":"fix","templateFieldSources":{"branchName":"default","commitMessage":"default","description":"default","title":"default"},"templateVariants":["updated-fix-title","pr-warning-shown","priorityScore"],"type":"auto","upgrade":["SNYK-RUBY-RACK-1061917"],"vulns":["SNYK-RUBY-RACK-1061917"],"patch":[],"isBreakingChange":false,"remediationStrategy":"vuln"}'

---------

Co-authored-by: snyk-bot <snyk-bot@snyk.io>
2024-09-16 09:44:07 +05:30
Sojan JoseandGitHub cb4ad28a13 chore: Auto resolution job for captain (#9898)
- Add a conversation auto-resolution job for the captain integration
2024-08-06 16:15:11 -07:00
PranavandGitHub ffd47081bd chore(cleanup): Delete sentiment feature (#9304)
- The feature is unused, removing it for now, will bring it back with better models later.
2024-04-25 22:49:10 -07:00
6956436a76 feat: refactor SLA evaluation logic (#9133)
* feat: update SLA evaluation logic

* chore: handle nrt

* chore: handle applied_sla status

* chore: refactor spec to bring down expecations in a single block

* chore: fix process_account_applied_sla spec

* chore: add spec to test multiple nrt misses

* feat: persist sla notifications

* feat: revert persist sla notifications

* chore: refactor sla_status to include active_with_misses

* chore: refactor spec

* Update evaluate_applied_sla_service.rb

* minor refactors

* clean up

* move notification related spec

* chore: refactor notifications spec to sla_event model

---------

Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
Co-authored-by: Sojan Jose <sojan@pepalo.com>
2024-03-29 02:01:43 +11:00
Sojan JoseandGitHub 41e269e873 feat(ee): Add reporting events for bots (#9027)
Added a new event conversation_bot_resolved and added a job to auto resolve the bot conversations if there was no activity for the last 1 hour.
2024-02-27 14:53:28 -08:00