From ef54f07d5b4f42605c2ac06c64d79f3a5c9b7abd Mon Sep 17 00:00:00 2001
From: Vinay Keerthi <11478411+stonecharioteer@users.noreply.github.com>
Date: Mon, 3 Nov 2025 20:03:47 +0530
Subject: [PATCH 1/3] feat: Add company backfill migration for existing
 contacts (Part 1) (#12657)

## Description

Implements company backfill migration infrastructure for existing
contacts. This is **Part 1 of 2** for the company model production
rollout as described in
[CW-5726](https://linear.app/chatwoot/issue/CW-5726/company-model-setting-it-up-on-production).

Creates jobs and services to associate existing contacts with companies
based on their email domains, filtering out free email providers (gmail,
yahoo, etc.) and disposable addresses.


**What's included:**
- Business email detector service with ValidEmail2 (uses
`disposable_domain?` to avoid DNS lookups)
- Per-account batch job to process contacts for one account
- Orchestrator job to iterate all accounts
- Rake task: `bundle exec rake companies:backfill`

~~*NOTE*: I'm using a hard-coded approach to determine if something is a
"business" email by filtering out emails that are usually personal. I've
also added domains that are common to some of our customers' regions.
This should be simpler. I looked into `Valid_Email2` and I couldn't find
anything to dictate whether an email is a personal email or a business
one. I don't think the approach used in the frontend is valid here.~~
UPDATE: Using `email_provider_info` gem instead.


**Pending - Part 2 (separate PR):** Real-time company creation for new
contacts

## Type of change

- [x] New feature (non-breaking change which adds functionality)

## How Has This Been Tested?

```bash
# Run all new tests
bundle exec rspec spec/enterprise/services/companies/business_email_detector_service_spec.rb \\
                   spec/enterprise/jobs/migration/company_account_batch_job_spec.rb \\
                   spec/enterprise/jobs/migration/company_backfill_job_spec.rb

# Run RuboCop
bundle exec rubocop enterprise/app/services/companies/business_email_detector_service.rb \\
                     enterprise/app/jobs/migration/company_account_batch_job.rb \\
                     enterprise/app/jobs/migration/company_backfill_job.rb \\
                     lib/tasks/companies.rake
```

**Performance optimization:**
- Uses `disposable_domain?` instead of `disposable?` to avoid DNS MX
lookups (discovered via tcpdump analysis - `disposable?` was making
network calls for every email, causing 100x slowdown)

## Checklist:

- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged and published in downstream
modules

---------

Co-authored-by: Sojan Jose <sojan@pepalo.com>
---
 Gemfile                                       |   1 +
 Gemfile.lock                                  |   2 +
 ...42_add_unique_index_to_companies_domain.rb |  16 +++
 db/schema.rb                                  |   2 +-
 .../migration/company_account_batch_job.rb    |  54 ++++++++
 .../jobs/migration/company_backfill_job.rb    |  17 +++
 enterprise/app/models/company.rb              |   7 +-
 .../business_email_detector_service.rb        |  19 +++
 lib/tasks/companies.rake                      |  12 ++
 .../company_account_batch_job_spec.rb         | 120 ++++++++++++++++++
 .../migration/company_backfill_job_spec.rb    |  31 +++++
 .../business_email_detector_service_spec.rb   |  99 +++++++++++++++
 12 files changed, 376 insertions(+), 4 deletions(-)
 create mode 100644 db/migrate/20251021082242_add_unique_index_to_companies_domain.rb
 create mode 100644 enterprise/app/jobs/migration/company_account_batch_job.rb
 create mode 100644 enterprise/app/jobs/migration/company_backfill_job.rb
 create mode 100644 enterprise/app/services/companies/business_email_detector_service.rb
 create mode 100644 lib/tasks/companies.rake
 create mode 100644 spec/enterprise/jobs/migration/company_account_batch_job_spec.rb
 create mode 100644 spec/enterprise/jobs/migration/company_backfill_job_spec.rb
 create mode 100644 spec/enterprise/services/companies/business_email_detector_service_spec.rb

diff --git a/Gemfile b/Gemfile
index 18442e3b0..abbd3332f 100644
--- a/Gemfile
+++ b/Gemfile
@@ -21,6 +21,7 @@ gem 'telephone_number'
 gem 'time_diff'
 gem 'tzinfo-data'
 gem 'valid_email2'
+gem 'email-provider-info'
 # compress javascript config.assets.js_compressor
 gem 'uglifier'
 ##-- used for single column multiple binary flags in notification settings/feature flagging --##
diff --git a/Gemfile.lock b/Gemfile.lock
index 2f4da34e3..99e75b33c 100644
--- a/Gemfile.lock
+++ b/Gemfile.lock
@@ -270,6 +270,7 @@ GEM
       concurrent-ruby (~> 1.0)
       http (>= 3.0)
       ruby2_keywords
+    email-provider-info (0.0.1)
     email_reply_trimmer (0.1.13)
     erubi (1.13.0)
     et-orbi (1.2.11)
@@ -1016,6 +1017,7 @@ DEPENDENCIES
   dotenv-rails (>= 3.0.0)
   down
   elastic-apm
+  email-provider-info
   email_reply_trimmer
   facebook-messenger
   factory_bot_rails (>= 6.4.3)
diff --git a/db/migrate/20251021082242_add_unique_index_to_companies_domain.rb b/db/migrate/20251021082242_add_unique_index_to_companies_domain.rb
new file mode 100644
index 000000000..2fb387242
--- /dev/null
+++ b/db/migrate/20251021082242_add_unique_index_to_companies_domain.rb
@@ -0,0 +1,16 @@
+class AddUniqueIndexToCompaniesDomain < ActiveRecord::Migration[7.1]
+  def up
+    remove_index :companies, name: 'index_companies_on_domain_and_account_id', if_exists: true
+
+    add_index :companies, [:account_id, :domain],
+              unique: true,
+              name: 'index_companies_on_account_and_domain',
+              where: 'domain IS NOT NULL'
+  end
+
+  def down
+    remove_index :companies, name: 'index_companies_on_account_and_domain', if_exists: true
+    add_index :companies, [:domain, :account_id],
+              name: 'index_companies_on_domain_and_account_id'
+  end
+end
diff --git a/db/schema.rb b/db/schema.rb
index 022a0101e..45165d5c5 100644
--- a/db/schema.rb
+++ b/db/schema.rb
@@ -577,8 +577,8 @@ ActiveRecord::Schema[7.1].define(version: 2025_10_22_152158) do
     t.bigint "account_id", null: false
     t.datetime "created_at", null: false
     t.datetime "updated_at", null: false
+    t.index ["account_id", "domain"], name: "index_companies_on_account_and_domain", unique: true, where: "(domain IS NOT NULL)"
     t.index ["account_id"], name: "index_companies_on_account_id"
-    t.index ["domain", "account_id"], name: "index_companies_on_domain_and_account_id"
     t.index ["name", "account_id"], name: "index_companies_on_name_and_account_id"
   end
 
diff --git a/enterprise/app/jobs/migration/company_account_batch_job.rb b/enterprise/app/jobs/migration/company_account_batch_job.rb
new file mode 100644
index 000000000..2e6c20f19
--- /dev/null
+++ b/enterprise/app/jobs/migration/company_account_batch_job.rb
@@ -0,0 +1,54 @@
+class Migration::CompanyAccountBatchJob < ApplicationJob
+  queue_as :low
+
+  def perform(account)
+    account.contacts
+           .where.not(email: nil)
+           .find_in_batches(batch_size: 1000) do |contact_batch|
+      process_contact_batch(contact_batch, account)
+    end
+  end
+
+  private
+
+  def process_contact_batch(contacts, account)
+    contacts.each do |contact|
+      next unless should_process?(contact)
+
+      company = find_or_create_company(contact, account)
+      # rubocop:disable Rails/SkipsModelValidations
+      contact.update_column(:company_id, company.id) if company
+      # rubocop:enable Rails/SkipsModelValidations
+    end
+  end
+
+  def should_process?(contact)
+    return false if contact.company_id.present?
+    return false if contact.email.blank?
+
+    Companies::BusinessEmailDetectorService.new(contact.email).perform
+  end
+
+  def find_or_create_company(contact, account)
+    domain = extract_domain(contact.email)
+    company_name = derive_company_name(contact, domain)
+
+    Company.find_or_create_by!(account: account, domain: domain) do |company|
+      company.name = company_name
+    end
+  rescue ActiveRecord::RecordNotUnique
+    # Race condition: Another job created it between our check and create
+    # just find the one that was created
+
+    Company.find_by(account: account, domain: domain)
+  end
+
+  def extract_domain(email)
+    email.split('@').last&.downcase
+  end
+
+  def derive_company_name(contact, domain)
+    contact.additional_attributes&.dig('company_name').presence ||
+      domain.split('.').first.tr('-_', ' ').titleize
+  end
+end
diff --git a/enterprise/app/jobs/migration/company_backfill_job.rb b/enterprise/app/jobs/migration/company_backfill_job.rb
new file mode 100644
index 000000000..db9a3370f
--- /dev/null
+++ b/enterprise/app/jobs/migration/company_backfill_job.rb
@@ -0,0 +1,17 @@
+class Migration::CompanyBackfillJob < ApplicationJob
+  queue_as :low
+
+  def perform
+    Rails.logger.info 'Starting company backfill migration...'
+    account_count = 0
+    Account.find_in_batches(batch_size: 100) do |accounts|
+      accounts.each do |account|
+        Rails.logger.info "Enqueuing company backfill for account #{account.id}"
+        Migration::CompanyAccountBatchJob.perform_later(account)
+        account_count += 1
+      end
+    end
+
+    Rails.logger.info "Company backfill migration complete. Enqueued jobs for #{account_count} accounts."
+  end
+end
diff --git a/enterprise/app/models/company.rb b/enterprise/app/models/company.rb
index 764cb2a9c..fde6cb122 100644
--- a/enterprise/app/models/company.rb
+++ b/enterprise/app/models/company.rb
@@ -12,9 +12,9 @@
 #
 # Indexes
 #
-#  index_companies_on_account_id             (account_id)
-#  index_companies_on_domain_and_account_id  (domain,account_id)
-#  index_companies_on_name_and_account_id    (name,account_id)
+#  index_companies_on_account_and_domain   (account_id,domain) UNIQUE WHERE (domain IS NOT NULL)
+#  index_companies_on_account_id           (account_id)
+#  index_companies_on_name_and_account_id  (name,account_id)
 #
 class Company < ApplicationRecord
   include Avatarable
@@ -24,6 +24,7 @@ class Company < ApplicationRecord
     with: /\A[a-zA-Z0-9]([a-zA-Z0-9-]*[a-zA-Z0-9])?(\.[a-zA-Z0-9]([a-zA-Z0-9-]*[a-zA-Z0-9])?)+\z/,
     message: I18n.t('errors.companies.domain.invalid')
   }
+  validates :domain, uniqueness: { scope: :account_id }, if: -> { domain.present? }
   validates :description, length: { maximum: Limits::COMPANY_DESCRIPTION_LENGTH_LIMIT }
 
   belongs_to :account
diff --git a/enterprise/app/services/companies/business_email_detector_service.rb b/enterprise/app/services/companies/business_email_detector_service.rb
new file mode 100644
index 000000000..422ef41c8
--- /dev/null
+++ b/enterprise/app/services/companies/business_email_detector_service.rb
@@ -0,0 +1,19 @@
+class Companies::BusinessEmailDetectorService
+  attr_reader :email
+
+  def initialize(email)
+    @email = email
+  end
+
+  def perform
+    return false if email.blank?
+
+    address = ValidEmail2::Address.new(email)
+    return false unless address.valid?
+    return false if address.disposable_domain?
+
+    provider = EmailProviderInfo.call(email)
+
+    provider.nil?
+  end
+end
diff --git a/lib/tasks/companies.rake b/lib/tasks/companies.rake
new file mode 100644
index 000000000..11fb5dc10
--- /dev/null
+++ b/lib/tasks/companies.rake
@@ -0,0 +1,12 @@
+namespace :companies do
+  desc 'Backfill companies from existing contact email domains'
+  task backfill: :environment do
+    puts 'Starting company backfill migration...'
+    puts 'This will process all accounts and create companies from contact email domains.'
+    puts 'The job will run in the background via Sidekiq'
+    puts ''
+    Migration::CompanyBackfillJob.perform_later
+    puts 'Company backfill job has been enqueued.'
+    puts 'Monitor progress in logs or Sidekiq dashboard.'
+  end
+end
diff --git a/spec/enterprise/jobs/migration/company_account_batch_job_spec.rb b/spec/enterprise/jobs/migration/company_account_batch_job_spec.rb
new file mode 100644
index 000000000..9a3e88c9d
--- /dev/null
+++ b/spec/enterprise/jobs/migration/company_account_batch_job_spec.rb
@@ -0,0 +1,120 @@
+require 'rails_helper'
+
+RSpec.describe Migration::CompanyAccountBatchJob, type: :job do
+  let(:account) { create(:account) }
+
+  describe '#perform' do
+    before do
+      # Stub EmailProvideInfo to control behavior in tests
+      allow(EmailProviderInfo).to receive(:call) do |email|
+        domain = email.split('@').last&.downcase
+        case domain
+        when 'gmail.com', 'yahoo.com', 'hotmail.com', 'uol.com.br'
+          'free_provider' # generic free provider name
+        end
+      end
+    end
+
+    context 'when contact has business email' do
+      let!(:contact) { create(:contact, account: account, email: 'user@acme.com') }
+
+      it 'creates a company and associates the contact' do
+        expect do
+          described_class.perform_now(account)
+        end.to change(Company, :count).by(1)
+        contact.reload
+        expect(contact.company).to be_present
+        expect(contact.company.domain).to eq('acme.com')
+        expect(contact.company.name).to eq('Acme')
+      end
+    end
+
+    context 'when contact has free email' do
+      let!(:contact) { create(:contact, account: account, email: 'user@gmail.com') }
+
+      it 'does not create a company' do
+        expect do
+          described_class.perform_now(account)
+        end.not_to change(Company, :count)
+        contact.reload
+        expect(contact.company_id).to be_nil
+      end
+    end
+
+    context 'when contact has company_name in additional_attributes' do
+      let!(:contact) do
+        create(:contact, account: account, email: 'user@acme.com', additional_attributes: { 'company_name' => 'Acme Corporation' })
+      end
+
+      it 'uses the saved company name' do
+        described_class.perform_now(account)
+        contact.reload
+        expect(contact.company.name).to eq('Acme Corporation')
+      end
+    end
+
+    context 'when contact already has a company' do
+      let!(:existing_company) { create(:company, account: account, domain: 'existing.com') }
+      let!(:contact) do
+        create(:contact, account: account, email: 'user@acme.com', company: existing_company)
+      end
+
+      it 'does not change the existing company' do
+        described_class.perform_now(account)
+        contact.reload
+        expect(contact.company_id).to eq(existing_company.id)
+      end
+    end
+
+    context 'when multiple contacts have the same domain' do
+      let!(:contact1) { create(:contact, account: account, email: 'user1@acme.com') }
+      let!(:contact2) { create(:contact, account: account, email: 'user2@acme.com') }
+
+      it 'creates only one company for the domain' do
+        expect do
+          described_class.perform_now(account)
+        end.to change(Company, :count).by(1)
+        contact1.reload
+        contact2.reload
+        expect(contact1.company_id).to eq(contact2.company_id)
+        expect(contact1.company.domain).to eq('acme.com')
+      end
+    end
+
+    context 'when contact has no email' do
+      let!(:contact) { create(:contact, account: account, email: nil) }
+
+      it 'skips the contact' do
+        expect do
+          described_class.perform_now(account)
+        end.not_to change(Company, :count)
+        contact.reload
+        expect(contact.company_id).to be_nil
+      end
+    end
+
+    context 'when processing large batch' do
+      before do
+        contacts_data = Array.new(2000) do |i|
+          {
+            account_id: account.id,
+            email: "user#{i}@company#{i % 100}.com",
+            name: "User #{i}",
+            created_at: Time.current,
+            updated_at: Time.current
+          }
+        end
+        # rubocop:disable Rails/SkipsModelValidations
+        Contact.insert_all(contacts_data)
+        # rubocop:enable Rails/SkipsModelValidations
+      end
+
+      it 'processes all contacts in batches' do
+        expect do
+          described_class.perform_now(account)
+        end.to change(Company, :count).by(100)
+        expect(account.contacts.where.not(company_id: nil).count).to eq(2000)
+      end
+    end
+  end
+end
diff --git a/spec/enterprise/jobs/migration/company_backfill_job_spec.rb b/spec/enterprise/jobs/migration/company_backfill_job_spec.rb
new file mode 100644
index 000000000..e60b84487
--- /dev/null
+++ b/spec/enterprise/jobs/migration/company_backfill_job_spec.rb
@@ -0,0 +1,31 @@
+require 'rails_helper'
+
+RSpec.describe Migration::CompanyBackfillJob, type: :job do
+  describe '#perform' do
+    it 'enqueues the job' do
+      expect { described_class.perform_later }
+        .to have_enqueued_job(described_class)
+        .on_queue('low')
+    end
+
+    context 'when accounts exist' do
+      let!(:account1) { create(:account) }
+      let!(:account2) { create(:account) }
+
+      it 'enqueues CompanyAccountBatchJob for each account' do
+        expect do
+          described_class.perform_now
+        end.to have_enqueued_job(Migration::CompanyAccountBatchJob)
+          .with(account1)
+          .and have_enqueued_job(Migration::CompanyAccountBatchJob)
+          .with(account2)
+      end
+    end
+
+    context 'when no accounts exist' do
+      it 'completes without error' do
+        expect { described_class.perform_now }.not_to raise_error
+      end
+    end
+  end
+end
diff --git a/spec/enterprise/services/companies/business_email_detector_service_spec.rb b/spec/enterprise/services/companies/business_email_detector_service_spec.rb
new file mode 100644
index 000000000..ceabfa905
--- /dev/null
+++ b/spec/enterprise/services/companies/business_email_detector_service_spec.rb
@@ -0,0 +1,99 @@
+require 'rails_helper'
+
+RSpec.describe Companies::BusinessEmailDetectorService, type: :service do
+  let(:service) { described_class.new(email) }
+
+  describe '#perform' do
+    context 'when email is from a business domain' do
+      let(:email) { 'user@acme.com' }
+      let(:valid_email_address) { instance_double(ValidEmail2::Address, valid?: true, disposable_domain?: false) }
+
+      before do
+        allow(ValidEmail2::Address).to receive(:new).with(email).and_return(valid_email_address)
+        allow(EmailProviderInfo).to receive(:call).with(email).and_return(nil)
+      end
+
+      it 'returns true' do
+        expect(service.perform).to be(true)
+      end
+    end
+
+    context 'when email is from gmail' do
+      let(:email) { 'user@gmail.com' }
+      let(:valid_email_address) { instance_double(ValidEmail2::Address, valid?: true, disposable_domain?: false) }
+
+      before do
+        allow(ValidEmail2::Address).to receive(:new).with(email).and_return(valid_email_address)
+        allow(EmailProviderInfo).to receive(:call).with(email).and_return('gmail')
+      end
+
+      it 'returns false' do
+        expect(service.perform).to be(false)
+      end
+    end
+
+    context 'when email is from Brazilian free provider' do
+      let(:email) { 'user@uol.com.br' }
+      let(:valid_email_address) { instance_double(ValidEmail2::Address, valid?: true, disposable_domain?: false) }
+
+      before do
+        allow(ValidEmail2::Address).to receive(:new).with(email).and_return(valid_email_address)
+        allow(EmailProviderInfo).to receive(:call).with(email).and_return('uol')
+      end
+
+      it 'returns false' do
+        expect(service.perform).to be(false)
+      end
+    end
+
+    context 'when email is disposable' do
+      let(:email) { 'user@mailinator.com' }
+      let(:disposable_email_address) { instance_double(ValidEmail2::Address, valid?: true, disposable_domain?: true) }
+
+      it 'returns false' do
+        allow(ValidEmail2::Address).to receive(:new).with(email).and_return(disposable_email_address)
+        expect(service.perform).to be(false)
+      end
+    end
+
+    context 'when email is invalid format' do
+      let(:email) { 'invalid-email' }
+      let(:invalid_email_address) { instance_double(ValidEmail2::Address, valid?: false) }
+
+      it 'returns false' do
+        allow(ValidEmail2::Address).to receive(:new).with(email).and_return(invalid_email_address)
+        expect(service.perform).to be(false)
+      end
+    end
+
+    context 'when email is nil' do
+      let(:email) { nil }
+
+      it 'remains false' do
+        expect(service.perform).to be(false)
+      end
+    end
+
+    context 'when email is empty string' do
+      let(:email) { '' }
+
+      it 'returns false' do
+        expect(service.perform).to be(false)
+      end
+    end
+
+    context 'when email domain is uppercase' do
+      let(:email) { 'user@GMAIL.COM' }
+      let(:valid_email_address) { instance_double(ValidEmail2::Address, valid?: true, disposable_domain?: false) }
+
+      before do
+        allow(ValidEmail2::Address).to receive(:new).with(email).and_return(valid_email_address)
+        allow(EmailProviderInfo).to receive(:call).with(email).and_return('gmail')
+      end
+
+      it 'returns false (case insensitive)' do
+        expect(service.perform).to be(false)
+      end
+    end
+  end
+end

From 1fbdd68222f3c44aaebbda50b80f64b82e680954 Mon Sep 17 00:00:00 2001
From: Vinay Keerthi <11478411+stonecharioteer@users.noreply.github.com>
Date: Mon, 3 Nov 2025 20:36:13 +0530
Subject: [PATCH 2/3] feat: Add company auto-association for contacts (CW-5726
 Part 2) (#12711)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

## Description

Implements real-time company auto-association for contacts based on
email domains. This is **Part 2** of the company model production
rollout (CW-5726).

**Task:**
- When a contact is created with a business email, automatically create
and associate a company from the email domain
- When a contact is updated with an email for the first time (email was
previously nil), associate with a company
- Preserve existing company associations when email changes to avoid
user confusion
- Skip free email providers and disposable domains

**Dependencies:**
⚠️ Requires PR #12657 (Part 1: Backfill migration) to be merged first

**Linear ticket:**
[CW-5726](https://linear.app/chatwoot/issue/CW-5726/company-model-setting-it-up-on-production)

## Type of change

- [x] New feature (non-breaking change which adds functionality)

## How Has This Been Tested?

- Service specs: Tests business email detection, company creation,
association logic, edge cases (existing companies, free emails, nil
emails)
- Integration specs: Tests full callback flow for contact create/update
scenarios
- All tests passing: 10 examples, 0 failures
- RuboCop: 0 offenses

## Checklist:

- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented on my code, particularly in hard-to-understand
areas
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged and published in downstream
modules (PR #12657 pending)

---------

Co-authored-by: Sojan Jose <sojan@pepalo.com>
---
 .../app/models/enterprise/concerns/contact.rb | 25 ++++++
 .../contacts/company_association_service.rb   | 46 ++++++++++
 .../company_account_batch_job_spec.rb         | 13 +++
 .../contact_company_association_spec.rb       | 61 ++++++++++++++
 .../company_association_service_spec.rb       | 83 +++++++++++++++++++
 5 files changed, 228 insertions(+)
 create mode 100644 enterprise/app/services/contacts/company_association_service.rb
 create mode 100644 spec/enterprise/models/contact_company_association_spec.rb
 create mode 100644 spec/enterprise/services/contacts/company_association_service_spec.rb

diff --git a/enterprise/app/models/enterprise/concerns/contact.rb b/enterprise/app/models/enterprise/concerns/contact.rb
index 9139fc67e..09885a947 100644
--- a/enterprise/app/models/enterprise/concerns/contact.rb
+++ b/enterprise/app/models/enterprise/concerns/contact.rb
@@ -2,5 +2,30 @@ module Enterprise::Concerns::Contact
   extend ActiveSupport::Concern
   included do
     belongs_to :company, optional: true
+
+    after_commit :associate_company_from_email,
+                 on: [:create, :update],
+                 if: :should_associate_company?
+  end
+
+  private
+
+  def should_associate_company?
+    # Only trigger if:
+    # 1. Contact has an email
+    # 2. Contact doesn't have a compan yet
+    # 3. Email was just set/changed
+    # 4. Email was previously nil (first time getting email)
+    email.present? &&
+      company_id.nil? &&
+      saved_change_to_email? &&
+      saved_change_to_email.first.nil?
+  end
+
+  def associate_company_from_email
+    Contacts::CompanyAssociationService.new.associate_company_from_email(self)
+  rescue StandardError => e
+    Rails.logger.error("Failed to associate company for contact #{id}: #{e.message}")
+    # Don't fail the contact save if the company association fails
   end
 end
diff --git a/enterprise/app/services/contacts/company_association_service.rb b/enterprise/app/services/contacts/company_association_service.rb
new file mode 100644
index 000000000..f2e2ffdd2
--- /dev/null
+++ b/enterprise/app/services/contacts/company_association_service.rb
@@ -0,0 +1,46 @@
+class Contacts::CompanyAssociationService
+  def associate_company_from_email(contact)
+    return nil if skip_association?(contact)
+
+    company = find_or_create_company(contact)
+    # rubocop:disable Rails/SkipsModelValidations
+    # Intentionally using update_column here to:
+    # 1. Avoid triggering callbacks
+    # 2. Improve performance (We're only setting company_id, no need for validation)
+    contact.update_column(:company_id, company.id) if company
+    # rubocop:enable Rails/SkipsModelValidations
+    company
+  end
+
+  private
+
+  def skip_association?(contact)
+    return true if contact.company_id.present?
+    return true if contact.email.blank?
+
+    detector = Companies::BusinessEmailDetectorService.new(contact.email)
+    return true unless detector.perform
+
+    false
+  end
+
+  def find_or_create_company(contact)
+    domain = extract_domain(contact.email)
+    company_name = derive_company_name(contact, domain)
+
+    Company.find_or_create_by!(account: contact.account, domain: domain) do |company|
+      company.name = company_name
+    end
+  rescue ActiveRecord::RecordNotUnique
+    # If another process created it first, just find that
+    Company.find_by(account: contact.account, domain: domain)
+  end
+
+  def extract_domain(email)
+    email.split('@').last&.downcase
+  end
+
+  def derive_company_name(contact, domain)
+    contact.additional_attributes&.dig('company_name') || domain.split('.').first.tr('-_', ' ').titleize
+  end
+end
diff --git a/spec/enterprise/jobs/migration/company_account_batch_job_spec.rb b/spec/enterprise/jobs/migration/company_account_batch_job_spec.rb
index 9a3e88c9d..499b96287 100644
--- a/spec/enterprise/jobs/migration/company_account_batch_job_spec.rb
+++ b/spec/enterprise/jobs/migration/company_account_batch_job_spec.rb
@@ -19,6 +19,12 @@ RSpec.describe Migration::CompanyAccountBatchJob, type: :job do
       let!(:contact) { create(:contact, account: account, email: 'user@acme.com') }
 
       it 'creates a company and associates the contact' do
+        # Clean up companies created by Part 2's callback
+        Company.delete_all
+        # rubocop:disable Rails/SkipsModelValidations
+        contact.update_column(:company_id, nil)
+        # rubocop:enable Rails/SkipsModelValidations
+
         expect do
           described_class.perform_now(account)
         end.to change(Company, :count).by(1)
@@ -71,6 +77,13 @@ RSpec.describe Migration::CompanyAccountBatchJob, type: :job do
       let!(:contact2) { create(:contact, account: account, email: 'user2@acme.com') }
 
       it 'creates only one company for the domain' do
+        # Clean up companies created by Part 2's callback
+        Company.delete_all
+        # rubocop:disable Rails/SkipsModelValidations
+        contact1.update_column(:company_id, nil)
+        contact2.update_column(:company_id, nil)
+        # rubocop:enable Rails/SkipsModelValidations
+
         expect do
           described_class.perform_now(account)
         end.to change(Company, :count).by(1)
diff --git a/spec/enterprise/models/contact_company_association_spec.rb b/spec/enterprise/models/contact_company_association_spec.rb
new file mode 100644
index 000000000..065d5bc97
--- /dev/null
+++ b/spec/enterprise/models/contact_company_association_spec.rb
@@ -0,0 +1,61 @@
+require 'rails_helper'
+
+RSpec.describe Contact, type: :model do
+  describe 'company auto-association' do
+    let(:account) { create(:account) }
+
+    context 'when creating a new contact with business email' do
+      it 'automatically creates and associates a company' do
+        expect do
+          create(:contact, email: 'john@acme.com', account: account)
+        end.to change(Company, :count).by(1)
+        contact = described_class.last
+        expect(contact.company).to be_present
+        expect(contact.company.domain).to eq('acme.com')
+      end
+
+      it 'does not create company for free email providers' do
+        expect do
+          create(:contact, email: 'john@gmail.com', account: account)
+        end.not_to change(Company, :count)
+      end
+    end
+
+    context 'when updating a contact to add email for first time' do
+      it 'creates and associates company' do
+        contact = create(:contact, email: nil, account: account)
+        expect do
+          contact.update(email: 'john@acme.com')
+        end.to change(Company, :count).by(1)
+        contact.reload
+        expect(contact.company.domain).to eq('acme.com')
+      end
+    end
+
+    context 'when updating a contact that already has a company' do
+      it 'does not change company when email changes' do
+        existing_company = create(:company, domain: 'oldcompany.com', account: account)
+        contact = create(:contact, email: 'john@oldcompany.com', company: existing_company, account: account)
+
+        expect do
+          contact.update(email: 'john@new_company.com')
+        end.not_to change(Company, :count)
+        contact.reload
+        expect(contact.company).to eq(existing_company)
+      end
+    end
+
+    context 'when multiple contacts share the same domain' do
+      it 'associates all contacts with the same company' do
+        contacts = ['john@acme.com', 'jane@acme.com', 'bob@acme.com']
+        contacts.each do |contact|
+          create(:contact, email: contact, account: account)
+        end
+
+        expect(Company.where(domain: 'acme.com', account: account).count).to eq(1)
+        company = Company.find_by(domain: 'acme.com', account: account)
+        expect(company.contacts.count).to eq(contacts.length)
+      end
+    end
+  end
+end
diff --git a/spec/enterprise/services/contacts/company_association_service_spec.rb b/spec/enterprise/services/contacts/company_association_service_spec.rb
new file mode 100644
index 000000000..ea9363bad
--- /dev/null
+++ b/spec/enterprise/services/contacts/company_association_service_spec.rb
@@ -0,0 +1,83 @@
+require 'rails_helper'
+
+RSpec.describe Contacts::CompanyAssociationService, type: :service do
+  let(:account) { create(:account) }
+  let(:service) { described_class.new }
+
+  describe '#associate_company_from_email' do
+    context 'when contact has business email and no company' do
+      it 'creates a new company and associates it' do
+        contact = create(:contact, email: 'john@acme.com', account: account, company_id: nil)
+        Company.delete_all # Delete any companies created by the callback
+        # rubocop:disable Rails/SkipsModelValidations
+        contact.update_column(:company_id, nil) # Delete the company association created by the callback
+        # rubocop:enable Rails/SkipsModelValidations
+
+        valid_email_address = instance_double(ValidEmail2::Address, valid?: true, disposable_domain?: false)
+        allow(ValidEmail2::Address).to receive(:new).with('john@acme.com').and_return(valid_email_address)
+        allow(EmailProviderInfo).to receive(:call).with('john@acme.com').and_return(nil)
+
+        expect do
+          service.associate_company_from_email(contact)
+        end.to change(Company, :count).by(1)
+
+        contact.reload
+        expect(contact.company).to be_present
+        expect(contact.company.domain).to eq('acme.com')
+        expect(contact.company.name).to eq('Acme')
+      end
+
+      it 'reuses existing company with same domain' do
+        existing_company = create(:company, domain: 'acme.com', account: account)
+        contact = create(:contact, email: 'john@acme.com', account: account, company_id: nil)
+        # rubocop:disable Rails/SkipsModelValidations
+        contact.update_column(:company_id, nil) # Delete the company association created by the callback
+        # rubocop:enable Rails/SkipsModelValidations
+
+        valid_email_address = instance_double(ValidEmail2::Address, valid?: true, disposable_domain?: false)
+        allow(ValidEmail2::Address).to receive(:new).with('john@acme.com').and_return(valid_email_address)
+        allow(EmailProviderInfo).to receive(:call).with('john@acme.com').and_return(nil)
+
+        expect do
+          service.associate_company_from_email(contact)
+        end.not_to change(Company, :count)
+
+        contact.reload
+        expect(contact.company).to eq(existing_company)
+      end
+    end
+
+    context 'when contact already has a company' do
+      it 'skips association and returns nil' do
+        existing_company = create(:company, account: account)
+        contact = create(:contact, email: 'john@acme.com', account: account, company_id: existing_company.id)
+        result = service.associate_company_from_email(contact)
+
+        expect(result).to be_nil
+        contact.reload
+        expect(contact.company).to eq(existing_company)
+      end
+    end
+
+    context 'when contact has free email provider' do
+      it 'skips association for email' do
+        contact = create(:contact, email: 'john@gmail.com', account: account, company_id: nil)
+        expect do
+          service.associate_company_from_email(contact)
+        end.not_to change(Company, :count)
+        contact.reload
+        expect(contact.company).to be_nil
+      end
+    end
+
+    context 'when contact has no email' do
+      it 'skips association' do
+        contact = create(:contact, email: nil, account: account, company_id: nil)
+
+        result = service.associate_company_from_email(contact)
+        expect(result).to be_nil
+        expect(contact.reload.company).to be_nil
+      end
+    end
+  end
+end

From d9b840f1615b44c5b84cba4b5585973f78e5b12f Mon Sep 17 00:00:00 2001
From: Vinay Keerthi <11478411+stonecharioteer@users.noreply.github.com>
Date: Tue, 4 Nov 2025 07:07:51 +0530
Subject: [PATCH 3/3] fix: Optimize Message search_data to prevent OpenSearch
 field explosion (#12786)

## Description

Refactored the `Message#search_data` method to prevent exceeding
OpenSearch's 1000 field limit during reindex operations.

**Problem:** The previous implementation serialized entire ActiveRecord
objects (Inbox, Sender, Conversation) with all their JSONB fields,
causing dynamic field explosion in OpenSearch. This resulted in
`Searchkick::ImportError` with "Limit of total fields [1000] has been
exceeded".

**Solution:** Whitelisted only necessary fields for search and
filtering, and flattened JSONB `custom_attributes` into key-value pair
arrays to prevent unbounded field creation.

Linked to: CW-5861

## Type of change

- [x] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality not to work as expected)
- [x] This change requires a documentation update

## How Has This Been Tested?

- Verified rubocop passes with no offenses
- Code review of search field usage from
`enterprise/app/services/enterprise/search_service.rb`
- Analyzed actual search queries to determine required indexed fields

**Still needed:**
- Full reindex test on staging/production environment
- Verify search functionality still works after reindex
- Confirm field count is under 1000 limit

## Changes Made

### Before
- Indexed 1000+ fields (entire AR objects with JSONB)
- `inbox` = full Inbox object (23+ fields + JSONB)
- `sender` = full Contact/User/AgentBot object (10+ fields + JSONB)
- `conversation` = full push_event_data
- Dynamic JSONB keys creating unlimited fields

### After
- ~35-40 controlled fields
- Whitelisted search fields: `content`, `attachment_transcribed_text`,
`email_subject`
- Filter fields: `account_id`, `inbox_id`, `conversation_id`,
`sender_id`, `sender_type`, etc.
- Flattened `custom_attributes`: `[{key, value, value_type}]` format
- Helper methods: `search_conversation_data`, `search_inbox_data`,
`search_sender_data`, `search_additional_data`

## Checklist:

- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged and published in downstream
modules

## Post-merge Steps

After merging, the following steps are required:

1. **Reindex all messages:**
   ```bash
   bundle exec rails runner "Message.reindex"
   ```

2. **Verify field count:**
   ```bash
   bundle exec rails runner "
     client = Searchkick.client
     index_name = Message.searchkick_index.name
     mapping = client.indices.get_mapping(index: index_name)
     fields = mapping.dig(index_name, 'mappings', 'properties')
     puts 'Total fields: ' + fields.keys.count.to_s
   "
   ```

3. **Test search functionality** to ensure queries still work as
expected

---------

Co-authored-by: Vishnu Narayanan <iamwishnu@gmail.com>
Co-authored-by: Pranav <pranav@chatwoot.com>
---
 app/models/message.rb                         | 13 +---
 .../messages/search_data_presenter.rb         | 58 ++++++++++++++
 .../messages/search_data_presenter_spec.rb    | 78 +++++++++++++++++++
 3 files changed, 140 insertions(+), 9 deletions(-)
 create mode 100644 app/presenters/messages/search_data_presenter.rb
 create mode 100644 spec/presenters/messages/search_data_presenter_spec.rb

diff --git a/app/models/message.rb b/app/models/message.rb
index 5f98493d0..2964d9286 100644
--- a/app/models/message.rb
+++ b/app/models/message.rb
@@ -153,15 +153,6 @@ class Message < ApplicationRecord
     merge_sender_attributes(data)
   end
 
-  def search_data
-    data = attributes.symbolize_keys
-    data[:conversation] = conversation.present? ? conversation_push_event_data : nil
-    data[:attachments] = attachments.map(&:push_event_data) if attachments.present?
-    data[:sender] = sender.push_event_data if sender
-    data[:inbox] = inbox
-    data
-  end
-
   def conversation_push_event_data
     {
       assignee_id: conversation.assignee_id,
@@ -259,6 +250,10 @@ class Message < ApplicationRecord
     true
   end
 
+  def search_data
+    Messages::SearchDataPresenter.new(self).search_data
+  end
+
   private
 
   def prevent_message_flooding
diff --git a/app/presenters/messages/search_data_presenter.rb b/app/presenters/messages/search_data_presenter.rb
new file mode 100644
index 000000000..dba0b9499
--- /dev/null
+++ b/app/presenters/messages/search_data_presenter.rb
@@ -0,0 +1,58 @@
+class Messages::SearchDataPresenter < SimpleDelegator
+  def search_data
+    {
+      **searchable_content,
+      **message_attributes,
+      **search_additional_data,
+      conversation: conversation_data
+    }
+  end
+
+  private
+
+  def searchable_content
+    {
+      content: content,
+      attachments: attachment_data,
+      content_attributes: content_attributes_data
+    }
+  end
+
+  def message_attributes
+    {
+      account_id: account_id,
+      inbox_id: inbox_id,
+      conversation_id: conversation_id,
+      message_type: message_type,
+      private: private,
+      created_at: created_at,
+      source_id: source_id,
+      sender_id: sender_id,
+      sender_type: sender_type
+    }
+  end
+
+  def attachment_data
+    attachments.filter_map do |a|
+      { transcribed_text: a.meta&.dig('transcribed_text') }
+    end.presence
+  end
+
+  def content_attributes_data
+    email_subject = content_attributes.dig(:email, :subject)
+    return {} if email_subject.blank?
+
+    { email: { subject: email_subject } }
+  end
+
+  def conversation_data
+    { id: conversation.display_id }
+  end
+
+  def search_additional_data
+    {
+      campaign_id: additional_attributes&.dig('campaign_id'),
+      automation_rule_id: content_attributes&.dig('automation_rule_id')
+    }
+  end
+end
diff --git a/spec/presenters/messages/search_data_presenter_spec.rb b/spec/presenters/messages/search_data_presenter_spec.rb
new file mode 100644
index 000000000..a5062086a
--- /dev/null
+++ b/spec/presenters/messages/search_data_presenter_spec.rb
@@ -0,0 +1,78 @@
+require 'rails_helper'
+
+RSpec.describe Messages::SearchDataPresenter do
+  let(:presenter) { described_class.new(message) }
+  let(:account) { create(:account) }
+  let(:inbox) { create(:inbox, account: account) }
+  let(:contact) { create(:contact, account: account) }
+  let(:conversation) { create(:conversation, account: account, inbox: inbox, contact: contact) }
+  let(:message) { create(:message, account: account, inbox: inbox, conversation: conversation, sender: contact) }
+
+  describe '#search_data' do
+    let(:expected_data) do
+      {
+        content: message.content,
+        account_id: message.account_id,
+        inbox_id: message.inbox_id,
+        conversation_id: message.conversation_id,
+        message_type: message.message_type,
+        private: message.private,
+        created_at: message.created_at,
+        source_id: message.source_id,
+        sender_id: message.sender_id,
+        sender_type: message.sender_type,
+        conversation: {
+          id: conversation.display_id
+        }
+      }
+    end
+
+    it 'returns search index payload with core fields' do
+      expect(presenter.search_data).to include(expected_data)
+    end
+
+    context 'with attachments' do
+      before do
+        attachment = message.attachments.new(account_id: message.account_id, file_type: :image)
+        attachment.file.attach(io: Rails.root.join('spec/assets/avatar.png').open, filename: 'avatar.png', content_type: 'image/png')
+        attachment.meta = { 'transcribed_text' => 'Hello world' }
+      end
+
+      it 'includes attachment transcriptions' do
+        attachments_data = presenter.search_data[:attachments]
+        expect(attachments_data).to be_an(Array)
+        expect(attachments_data.first).to include(transcribed_text: 'Hello world')
+      end
+    end
+
+    context 'with email content attributes' do
+      before do
+        message.update(
+          content_attributes: { email: { subject: 'Test Subject' } }
+        )
+      end
+
+      it 'includes email subject' do
+        content_attrs = presenter.search_data[:content_attributes]
+        expect(content_attrs[:email][:subject]).to eq('Test Subject')
+      end
+    end
+
+    context 'with campaign and automation data' do
+      before do
+        message.update(
+          additional_attributes: { 'campaign_id' => '123' },
+          content_attributes: { 'automation_rule_id' => '456' }
+        )
+      end
+
+      it 'includes campaign_id' do
+        expect(presenter.search_data[:campaign_id]).to eq('123')
+      end
+
+      it 'includes automation_rule_id' do
+        expect(presenter.search_data[:automation_rule_id]).to eq('456')
+      end
+    end
+  end
+end