# Pull Request Template
## Description
skip documents that fail with ActiveRecord errors possibly due to
stale/corrupt data and not crash scheduler
How did we find out about this error?
before October 28th, 2025, we did not have url normalisation.
so we had document rows as:
id: 123 `https://example.com` status: `in_progress` --> likely stuck
crawl
id 234: `https://example.com/` status: `available`
When the schedule sync job ran, it ran an `document.update!(sync_status:
:syncing, last_sync_attempted_at: Time.current)` on the 234 one since it
was `available`
now `update!` runs `before_validation :normalize_external_link`
so `https://example.com/` became `https://example.com`
which invalidated:
`validates :external_link, uniqueness: { scope: :assistant_id }`
so the scheduler crashed.
This PR logs the skipped ones with their errors and continues to pick
other documents to scheduler doesn't crash
## Type of change
- [x] Bug fix (non-breaking change which fixes an issue)
## How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide
instructions so we can reproduce. Please also list any relevant details
for your test configuration.
spec
## Checklist:
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] Any dependent changes have been merged and published in downstream
modules
