akadmin/chatwoot
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

feat: remove scratch files

Browse Source
This commit is contained in:
Shivam Mishra
2026-06-30 15:22:12 +05:30
parent 729aca7b64
commit 5c39d896d9
2 changed files with 214 additions and 224 deletions
Download Patch File Download Diff File Expand all files Collapse all files
queries.md
+214
View File
@@ -0,0 +1,214 @@
# Captain Overview — effective SQL per stat
These are the queries `Captain::AssistantStatsBuilder` effectively runs for
`account_id = 1`, `assistant_id = 1`, over a **30-day current window**
(`created_at >= NOW() - INTERVAL '30 days'`). Each metric runs once for the
current window and once for the previous equal-length window
(`NOW() - INTERVAL '60 days'` to `NOW() - INTERVAL '30 days'`); only the current
window is shown below. Swap the interval bounds for the previous window.
Knowledge stats are point-in-time (no window, no previous).
---
## 1. Conversations handled
Distinct conversations the assistant authored any message in.
```sql
SELECT COUNT(DISTINCT conversation_id)
FROM messages
WHERE account_id = 1
AND sender_type = 'Captain::Assistant'
AND sender_id = 1
AND created_at >= NOW() - INTERVAL '30 days';
```
Indexes that matter: `messages (account_id, ...)`, plus the `sender_type/sender_id`
filter. This is the "handled set" reused as a subquery by stats 2 and 3.
---
## 2 & 3. Auto-resolution rate + handoff rate (combined)
`auto_resolution = resolved / handled * 100` and `handoff = handoff / handled * 100`
share the same handled-set subquery, so they fold into a single pass with
conditional aggregation. This runs the (costly) `messages` handled-set subquery
**once** and scans `reporting_events` once, instead of twice each.
```sql
SELECT
COUNT(DISTINCT conversation_id)
FILTER (WHERE name IN ('conversation_captain_inference_resolved',
'conversation_bot_resolved')) AS resolved,
COUNT(DISTINCT conversation_id)
FILTER (WHERE name IN ('conversation_captain_inference_handoff',
'conversation_bot_handoff')) AS handoff
FROM reporting_events
WHERE account_id = 1
AND name IN ('conversation_captain_inference_resolved', 'conversation_bot_resolved',
'conversation_captain_inference_handoff', 'conversation_bot_handoff')
AND conversation_id IN (
SELECT conversation_id
FROM messages
WHERE account_id = 1
AND sender_type = 'Captain::Assistant'
AND sender_id = 1
AND created_at >= NOW() - INTERVAL '30 days'
);
```
The rates are still computed in Ruby (`resolved / handled`, `handoff / handled`),
so divide-by-zero guarding stays in the builder. The two rates do **not** sum to
100%: a handled conversation may carry neither event (still open) or both
(handed off then later resolved).
Note: the event rows themselves are **not** date-filtered; the window is applied
via the handled-set subquery (the conversations the assistant touched in the
window).
### Optional: fold in "handled" too (one query for all three)
Materialize the handled set once in a CTE and derive all three numbers from it.
Useful because the page also displays `handled`, and every metric runs for both
the current and previous window.
```sql
WITH handled AS (
SELECT DISTINCT conversation_id
FROM messages
WHERE account_id = 1
AND sender_type = 'Captain::Assistant'
AND sender_id = 1
AND created_at >= NOW() - INTERVAL '30 days'
)
SELECT
(SELECT COUNT(*) FROM handled) AS handled,
COUNT(DISTINCT re.conversation_id)
FILTER (WHERE re.name IN ('conversation_captain_inference_resolved',
'conversation_bot_resolved')) AS resolved,
COUNT(DISTINCT re.conversation_id)
FILTER (WHERE re.name IN ('conversation_captain_inference_handoff',
'conversation_bot_handoff')) AS handoff
FROM reporting_events re
JOIN handled h ON h.conversation_id = re.conversation_id
WHERE re.account_id = 1
AND re.name IN ('conversation_captain_inference_resolved', 'conversation_bot_resolved',
'conversation_captain_inference_handoff', 'conversation_bot_handoff');
```
Keep `handled` sourced from the CTE count (not `COUNT(*)` over the join), so
conversations with no event row still count toward `handled`.
---
## 4. Hours saved
`public_outgoing_count * avg_reply_time_seconds / 3600`, rounded. Two queries.
Public outgoing replies the assistant sent:
```sql
SELECT COUNT(*)
FROM messages
WHERE account_id = 1
AND sender_type = 'Captain::Assistant'
AND sender_id = 1
AND message_type = 1 -- outgoing
AND private = false
AND created_at >= NOW() - INTERVAL '30 days';
```
Average reply time across the account in the window (seconds):
```sql
SELECT AVG(value)
FROM reporting_events
WHERE account_id = 1
AND name = 'reply_time'
AND created_at >= NOW() - INTERVAL '30 days';
```
---
## 5. Conversation depth (messages / conversation)
`public_outgoing_count / distinct_conversations`. Reuses query 4's count plus:
```sql
SELECT COUNT(DISTINCT conversation_id)
FROM messages
WHERE account_id = 1
AND sender_type = 'Captain::Assistant'
AND sender_id = 1
AND message_type = 1 -- outgoing
AND private = false
AND created_at >= NOW() - INTERVAL '30 days';
```
---
## 6. Reopen-after-resolve rate
`reopened / resolved * 100`, where "resolved" is the inbox-based Captain-resolved
set. Two queries.
Resolved (inbox-scoped) in the window:
```sql
SELECT COUNT(DISTINCT conversation_id)
FROM reporting_events
WHERE account_id = 1
AND name = 'conversation_captain_inference_resolved'
AND inbox_id IN (
SELECT inbox_id FROM captain_inboxes WHERE captain_assistant_id = 1
)
AND created_at >= NOW() - INTERVAL '30 days';
```
Of those, the ones later reopened (`conversation_opened` with `value > 0`):
```sql
SELECT COUNT(DISTINCT conversation_id)
FROM reporting_events
WHERE account_id = 1
AND name = 'conversation_opened'
AND value > 0
AND conversation_id IN (
SELECT conversation_id
FROM reporting_events
WHERE account_id = 1
AND name = 'conversation_captain_inference_resolved'
AND inbox_id IN (
SELECT inbox_id FROM captain_inboxes WHERE captain_assistant_id = 1
)
AND created_at >= NOW() - INTERVAL '30 days'
);
```
Note: the `conversation_opened` events are **not** date-filtered; a reopen counts
whenever it happened, as long as the resolve fell in the window.
---
## 7. Knowledge (point-in-time, no window)
Approved / pending FAQ counts:
```sql
SELECT status, COUNT(*)
FROM captain_assistant_responses
WHERE assistant_id = 1
GROUP BY status;
-- status: 0 = pending, 1 = approved
```
Document count:
```sql
SELECT COUNT(*)
FROM captain_documents
WHERE assistant_id = 1;
```
`coverage = approved / (approved + pending) * 100`, computed in Ruby.
stats.md
-224
View File
@@ -1,224 +0,0 @@
# Captain Assistant Metrics — Exploration
Candidate metrics for a per-assistant overview page, sourced from `messages`,
`conversations`, `reporting_events`, and the `captain_*` tables.
## Grounding facts
- **Assistant → messages**: `Captain::Assistant has_many :messages, as: :sender` → Captain
messages are rows in `messages` with `sender_type = 'Captain::Assistant'`,
`sender_id = <assistant.id>`. Most reliable per-assistant attribution.
- **Assistant → inboxes → conversations**: `captain_inboxes(captain_assistant_id, inbox_id)`.
- **Reporting events** (in `reporting_events`, scoped by `conversation_id`/`inbox_id`, no
assistant column): `conversation_captain_inference_resolved`,
`conversation_captain_inference_handoff`, `conversation_bot_resolved`,
`conversation_bot_handoff`, `conversation_resolved`, `conversation_opened`
(value > 0 = a reopen).
- **Enums**: `messages.message_type` → `incoming:0, outgoing:1`; `conversations.status` →
`open:0, resolved:1, pending:2, snoozed:3`; `captain_assistant_responses.status` →
`pending:0, approved:1`.
All queries take `:assistant_id`, `:account_id`, and a `:start`/`:end` window.
---
## Conversations Handled
Definition: How many distinct conversations this assistant actually participated in (sent at
least one message). Your denominator for every rate below, and the headline "is anyone using
this assistant" volume number.
```sql
SELECT COUNT(DISTINCT m.conversation_id) AS conversations_handled
FROM messages m
WHERE m.account_id = :account_id
AND m.sender_type = 'Captain::Assistant'
AND m.sender_id = :assistant_id
AND m.created_at BETWEEN :start AND :end;
```
## Auto-Resolution (Deflection) Rate
Definition: Of the conversations Captain handled, the share it closed on its own with no human
reply. This is the core ROI/deflection signal — higher means more tickets the team never had
to touch.
```sql
WITH handled AS (
SELECT DISTINCT conversation_id
FROM messages
WHERE account_id = :account_id
AND sender_type = 'Captain::Assistant'
AND sender_id = :assistant_id
AND created_at BETWEEN :start AND :end
)
SELECT
COUNT(DISTINCT re.conversation_id)::float
/ NULLIF((SELECT COUNT(*) FROM handled), 0) AS auto_resolution_rate
FROM reporting_events re
JOIN handled h ON h.conversation_id = re.conversation_id
WHERE re.account_id = :account_id
AND re.name IN ('conversation_captain_inference_resolved',
'conversation_bot_resolved');
```
## Handoff Rate
Definition: Of the conversations Captain handled, the share it escalated to a human (either an
explicit handoff tool call mid-chat, or the auto-resolve job deciding the customer still needs
clarification). Inverse signal to deflection — tells you how often Captain hit its limits.
```sql
WITH handled AS (
SELECT DISTINCT conversation_id
FROM messages
WHERE account_id = :account_id
AND sender_type = 'Captain::Assistant'
AND sender_id = :assistant_id
AND created_at BETWEEN :start AND :end
)
SELECT
COUNT(DISTINCT re.conversation_id)::float
/ NULLIF((SELECT COUNT(*) FROM handled), 0) AS handoff_rate
FROM reporting_events re
JOIN handled h ON h.conversation_id = re.conversation_id
WHERE re.account_id = :account_id
AND re.name IN ('conversation_captain_inference_handoff',
'conversation_bot_handoff');
```
## Reopen-After-Auto-Resolve Rate (premature closures)
Definition: Of the conversations Captain auto-resolved, the share that got reopened afterwards.
A quality/over-eagerness signal — high values mean Captain is closing tickets the customer
wasn't actually done with. (`conversation_opened` with `value > 0` is a reopen, not the first
open.)
```sql
WITH resolved AS (
SELECT DISTINCT re.conversation_id
FROM reporting_events re
JOIN captain_inboxes ci ON ci.inbox_id = re.inbox_id
WHERE re.account_id = :account_id
AND ci.captain_assistant_id = :assistant_id
AND re.name = 'conversation_captain_inference_resolved'
AND re.created_at BETWEEN :start AND :end
)
SELECT
COUNT(DISTINCT o.conversation_id)::float
/ NULLIF((SELECT COUNT(*) FROM resolved), 0) AS reopen_rate
FROM reporting_events o
JOIN resolved r ON r.conversation_id = o.conversation_id
WHERE o.account_id = :account_id
AND o.name = 'conversation_opened'
AND o.value > 0;
```
## Flagged-Response Rate
Definition: How often human agents reported a Captain message as bad
(`captain_message_reports`), as a share of the public answers Captain produced. A direct
human-in-the-loop quality score — rising values mean agents are losing trust in the answers.
```sql
WITH captain_msgs AS (
SELECT id
FROM messages
WHERE account_id = :account_id
AND sender_type = 'Captain::Assistant'
AND sender_id = :assistant_id
AND message_type = 1
AND private = false
AND created_at BETWEEN :start AND :end
)
SELECT
(SELECT COUNT(*) FROM captain_message_reports r
WHERE r.message_id IN (SELECT id FROM captain_msgs))::float
/ NULLIF((SELECT COUNT(*) FROM captain_msgs), 0) AS flagged_rate;
```
## Conversation Depth (messages per handled conversation)
Definition: Average number of public replies Captain sends per conversation it handles. Low
(~1) = quick one-shot answers; high = Captain is grinding through long back-and-forths, which
often correlates with the cases it ends up handing off.
```sql
SELECT
COUNT(*)::float / NULLIF(COUNT(DISTINCT conversation_id), 0) AS msgs_per_conversation
FROM messages
WHERE account_id = :account_id
AND sender_type = 'Captain::Assistant'
AND sender_id = :assistant_id
AND message_type = 1
AND private = false
AND created_at BETWEEN :start AND :end;
```
## Hours Saved
Definition: A headline "what did Captain save us" number for execs: the volume of replies Captain
sent multiplied by the team's average response time. Reads as agent time deflected.
Caveat (important): the multiplier here is the team's average *response time* (`first_response` /
`reply_time`), which is how long a customer *waited*, not how long an agent actually *worked* on a
reply. Wait time is typically much larger than handling time, so this number overstates true labor
saved by a wide margin and should be presented as a directional/estimate figure, not measured
labor. Counting every Captain message (including ones in conversations later handed off to a human)
also inflates it. See the conversation that produced this file for the more conservative
"deflected conversations × handling time" alternative if a defensible number is needed.
```sql
SELECT
(
SELECT COUNT(*)
FROM messages m
WHERE m.account_id = :account_id
AND m.sender_type = 'Captain::Assistant'
AND m.sender_id = :assistant_id
AND m.message_type = 1
AND m.private = false
AND m.created_at BETWEEN :start AND :end
)
* (
SELECT AVG(re.value)
FROM reporting_events re
WHERE re.account_id = :account_id
AND re.name = 'reply_time'
AND re.created_at BETWEEN :start AND :end
) / 3600.0 AS hours_saved;
```
## Knowledge Base Coverage
Definition: How much knowledge backs this assistant — approved vs. still-pending FAQ responses,
plus synced documents. Not a performance metric but the leading indicator: thin/low-approval
knowledge usually explains a low deflection rate. (Time-independent; it's the assistant's
current state.)
```sql
SELECT
COUNT(*) FILTER (WHERE status = 1) AS approved_responses,
COUNT(*) FILTER (WHERE status = 0) AS pending_responses,
(SELECT COUNT(*) FROM captain_documents d
WHERE d.assistant_id = :assistant_id) AS documents
FROM captain_assistant_responses
WHERE assistant_id = :assistant_id;
```
---
## Notes for building the overview page
- **Two attribution paths exist** and answer slightly different questions. Message-based
(`sender_id = assistant`) = "conversations Captain actually engaged." Inbox-based
(`captain_inboxes`) = "conversations that flowed through Captain's inbox, engaged or not."
Message-based is used for the handled/rate metrics (tighter); inbox-based only for the reopen
metric since `inference_resolved` is emitted by the auto-resolve job. If an inbox has exactly
one assistant, the two converge.
- **`reporting_events` has no `assistant_id`** — everything is joined back through
`conversation_id` or `inbox_id`. Fine today, but multi-assistant-per-inbox support would want
assistant attribution stamped on the event at write time.
- **`value` on the captain inference events** is "seconds from conversation creation to the
event" (see `create_captain_inference_event`), so time-to-resolve / time-to-handoff are
almost free if you want a couple more timing metrics.