fix: captain auto sync scheduler resilience (#14379)

# Pull Request Template

## Description

skip documents that fail with ActiveRecord errors possibly due to
stale/corrupt data and not crash scheduler

How did we find out about this error?

before October 28th, 2025, we did not have url normalisation.
so we had document rows as: 
id: 123 `https://example.com` status: `in_progress` --> likely stuck
crawl
id 234: `https://example.com/` status: `available` 

When the schedule sync job ran, it ran an `document.update!(sync_status:
:syncing, last_sync_attempted_at: Time.current)` on the 234 one since it
was `available`

now `update!` runs `before_validation :normalize_external_link`
so `https://example.com/` became `https://example.com`

which invalidated:
`validates :external_link, uniqueness: { scope: :assistant_id }`

so the scheduler crashed.

This PR logs the skipped ones with their errors and continues to pick
other documents to scheduler doesn't crash

## Type of change

- [x] Bug fix (non-breaking change which fixes an issue)

## How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide
instructions so we can reproduce. Please also list any relevant details
for your test configuration.
spec

## Checklist:

- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] Any dependent changes have been merged and published in downstream
modules

spec/enterprise/jobs/captain/documents/schedule_syncs_job_spec.rb

@@ -122,6 +122,67 @@ RSpec.describe Captain::Documents::ScheduleSyncsJob, type: :job do
       expect { described_class.new.perform }
         .to have_enqueued_job(Captain::Documents::PerformSyncJob).with(document)
     end
+
+    it 'skips invalid legacy documents without counting them against the account cap' do
+      stub_const("#{described_class}::PER_ACCOUNT_HOURLY_CAP", 1)
+      create(
+        :captain_document,
+        assistant: assistant,
+        account: account,
+        status: :in_progress,
+        content: nil,
+        external_link: 'https://example.com'
+      )
+      invalid_document = build(
+        :captain_document,
+        assistant: assistant,
+        account: account,
+        status: :available,
+        sync_status: :synced,
+        last_synced_at: 2.days.ago,
+        last_sync_attempted_at: 2.days.ago,
+        external_link: 'https://example.com/'
+      )
+      invalid_document.save!(validate: false)
+      valid_document = create(:captain_document, assistant: assistant, account: account, status: :available)
+      valid_document.update!(sync_status: :synced, last_synced_at: 2.days.ago, last_sync_attempted_at: 2.days.ago)
+      clear_enqueued_jobs
+
+      expect { described_class.new.perform }.not_to raise_error
+      expect(Captain::Documents::PerformSyncJob).not_to have_been_enqueued.with(invalid_document)
+      expect(Captain::Documents::PerformSyncJob).to have_been_enqueued.with(valid_document)
+    end
+
+    it 'keeps paging due documents when invalid documents fill the first batch' do
+      stub_const("#{described_class}::PER_ACCOUNT_HOURLY_CAP", 1)
+      stub_const("#{described_class}::DUE_DOCUMENT_BATCH_SIZE", 1)
+      create(
+        :captain_document,
+        assistant: assistant,
+        account: account,
+        status: :in_progress,
+        content: nil,
+        external_link: 'https://example.com'
+      )
+      invalid_document = build(
+        :captain_document,
+        assistant: assistant,
+        account: account,
+        status: :available,
+        sync_status: :synced,
+        last_synced_at: 2.days.ago,
+        last_sync_attempted_at: 3.days.ago,
+        external_link: 'https://example.com/'
+      )
+      invalid_document.save!(validate: false)
+      valid_document = create(:captain_document, assistant: assistant, account: account, status: :available)
+      valid_document.update!(sync_status: :synced, last_synced_at: 2.days.ago, last_sync_attempted_at: 2.days.ago)
+      clear_enqueued_jobs
+
+      described_class.new.perform
+
+      expect(Captain::Documents::PerformSyncJob).to have_been_enqueued.with(valid_document)
+    end
   end
 
   context 'when more documents are due than the account cap allows' do