Shivam MishraandGitHub
73ba0b26e5
Instagram webhooks use a Redis mutex to protect the first-message path
for a contact/account pair. When multiple webhook events for the same
Instagram contact arrive at nearly the same time, they can all enter the
“find or create” flow together.
There are two pieces involved:
`ContactInbox` maps the external Instagram scoped user id to a Chatwoot
contact inside an inbox. That side already has a unique `(inbox_id,
source_id)` index and retry handling, so duplicate contact-inbox
creation is mostly protected at the database layer.
`Conversation` creation is more fragile. The message builder looks for
an existing active conversation for the contact/inbox and creates one
when none is found. If two first-message jobs run concurrently, both can
see “no active conversation yet” and both can create a conversation. The
mutex exists to bump those jobs apart long enough for the first one to
create the conversation, so the next one appends to it instead of
creating a duplicate.
In production, the old behavior could turn that race dampener into a
drop path. Once `Webhooks::InstagramEventsJob` exhausted its lock
retries, ActiveJob stopped retrying with
`MutexApplicationJob::LockAcquisitionError`. Since ActiveJob retries are
not FIFO, later jobs could still win the lock while older jobs kept
getting deferred until exhaustion.
## Mutex Application Job Changes
This adds `retry_on_lock_conflict` as a small wrapper around the
existing `retry_on LockAcquisitionError` pattern.
The new API keeps the default behavior intact, but lets jobs explicitly
define what should happen after lock retry exhaustion:
```ruby
retry_on_lock_conflict wait: 1.second,
attempts: 3,
on_exhaustion: :process_without_lock
```
The fallback receives the original job arguments, so job classes do not
need to reach into ActiveJob internals like `arguments.first`.
## Instagram Fallback
Instagram now processes the webhook payload even after the mutex retry
window is exhausted.
This matches what the mutex was meant to do in the first place: bump
concurrent events apart long enough to avoid duplicate conversation
creation, not permanently block or drop webhook events. If a job cannot
acquire the lock after a few retries, it continues through the normal
Instagram processing path without the mutex.
## Retry And Lock Tuning
The Instagram lock retry window now uses deterministic backoff:
```ruby
retry_on_lock_conflict wait: ->(executions) { executions.seconds },
attempts: 3,
on_exhaustion: :process_without_lock
```
The lock TTL is also set explicitly:
```ruby
with_lock(key, 3.seconds)
```
This matters because Rails treats `attempts` as total executions, not
retries after the first execution. With a fixed `wait: 1.second` and
`attempts: 3`, the exhaustion handler can run roughly two seconds after
the first lock conflict. That is earlier than a 3-second lock TTL, so
the fallback could process without the lock while the original mutex is
still valid. That would reopen the duplicate-conversation race the lock
is meant to dampen.
Using a proc wait makes the timing predictable and avoids Rails jitter
for this retry path. The first conflict waits about 1 second, the second
waits about 2 seconds, and the final attempt happens around the 3-second
mark. That lines the retry window up with the Redis lock TTL before
falling back to `process_without_lock`.
The protected race window is intentionally small. `ContactInbox`
creation already has database uniqueness protection, while conversation
creation is the softer `find active conversation || create` path. We
only need to give the first job enough time to create the conversation
state that later jobs should reuse.
The mutex still does not preserve message order, and ActiveJob retries
are not FIFO. A longer retry window would mostly add latency for hot
Instagram contacts without making ordering more correct. After the lock
TTL has elapsed, processing without the lock is the better tradeoff than
dead-lettering customer messages.
Chatwoot
The modern customer support platform, an open-source alternative to Intercom, Zendesk, Salesforce Service Cloud etc.
Chatwoot is the modern, open-source, and self-hosted customer support platform designed to help businesses deliver exceptional customer support experience. Built for scale and flexibility, Chatwoot gives you full control over your customer data while providing powerful tools to manage conversations across channels.
✨ Captain – AI Agent for Support
Supercharge your support with Captain, Chatwoot’s AI agent. Captain helps automate responses, handle common queries, and reduce agent workload—ensuring customers get instant, accurate answers. With Captain, your team can focus on complex conversations while routine questions are resolved automatically. Read more about Captain here.
💬 Omnichannel Support Desk
Chatwoot centralizes all customer conversations into one powerful inbox, no matter where your customers reach out from. It supports live chat on your website, email, Facebook, Instagram, Twitter, WhatsApp, Telegram, Line, SMS etc.
📚 Help center portal
Publish help articles, FAQs, and guides through the built-in Help Center Portal. Enable customers to find answers on their own, reduce repetitive queries, and keep your support team focused on more complex issues.
🗂️ Other features
Collaboration & Productivity
- Private Notes and @mentions for internal team discussions.
- Labels to organize and categorize conversations.
- Keyboard Shortcuts and a Command Bar for quick navigation.
- Canned Responses to reply faster to frequently asked questions.
- Auto-Assignment to route conversations based on agent availability.
- Multi-lingual Support to serve customers in multiple languages.
- Custom Views and Filters for better inbox organization.
- Business Hours and Auto-Responders to manage response expectations.
- Teams and Automation tools for scaling support workflows.
- Agent Capacity Management to balance workload across the team.
Customer Data & Segmentation
- Contact Management with profiles and interaction history.
- Contact Segments and Notes for targeted communication.
- Campaigns to proactively engage customers.
- Custom Attributes for storing additional customer data.
- Pre-Chat Forms to collect user information before starting conversations.
Integrations
- Slack Integration to manage conversations directly from Slack.
- Dialogflow Integration for chatbot automation.
- Dashboard Apps to embed internal tools within Chatwoot.
- Shopify Integration to view and manage customer orders right within Chatwoot.
- Use Google Translate to translate messages from your customers in realtime.
- Create and manage Linear tickets within Chatwoot.
Reports & Insights
- Live View of ongoing conversations for real-time monitoring.
- Conversation, Agent, Inbox, Label, and Team Reports for operational visibility.
- CSAT Reports to measure customer satisfaction.
- Downloadable Reports for offline analysis and reporting.
Documentation
Detailed documentation is available at chatwoot.com/help-center.
Translation process
The translation process for Chatwoot web and mobile app is managed at https://translate.chatwoot.com using Crowdin. Please read the translation guide for contributing to Chatwoot.
Branching model
We use the git-flow branching model. The base branch is develop.
If you are looking for a stable version, please use the master or tags labelled as v1.x.x.
Deployment
Heroku one-click deploy
Deploying Chatwoot to Heroku is a breeze. It's as simple as clicking this button:
Follow this link to understand setting the correct environment variables for the app to work with all the features. There might be breakages if you do not set the relevant environment variables.
DigitalOcean 1-Click Kubernetes deployment
Chatwoot now supports 1-Click deployment to DigitalOcean as a kubernetes app.
Other deployment options
For other supported options, checkout our deployment page.
Security
Looking to report a vulnerability? Please refer our SECURITY.md file.
Community
If you need help or just want to hang out, come, say hi on our Discord server.
Contributors
Thanks goes to all these wonderful people:
Chatwoot © 2017-2026, Chatwoot Inc - Released under the MIT License.