Changes searchkick callback behavior to check `should_index?` before
creating reindex jobs, preventing unnecessary job creation for messages
that don't need indexing (activity messages, unpaid accounts, etc.).
Previously, `callbacks: :async` created reindex jobs for all messages
(~5,100/min or 7.3M/day in production), which were then filtered by
`should_index?` inside the job worker - resulting in 98% wasted jobs
, Redis memory pressure and avoidable p0 alerts.
Now, `should_index?` is checked before job creation via `after_commit`
callback, reducing job creation to actual incoming/outgoing
messages from paid accounts.
Changes:
- Disable automatic searchkick callbacks
- Add manual `after_commit` callback with `should_index?` condition
- Add specs to verify callback behavior
Expected impact:
- 98% reduction in sidekiq job creation (~7.3M → ~150K jobs/day)
- Reduced redis memory usage
- Same async indexing behavior for eligible messages
Vishnu Narayanan
c10d39f42e