This PR adds a Captain Assistant **Overview** page to show some KPI metrics (conversations handled, auto-resolution, handoff, hours saved, reopen-after-resolve, conversation depth) with trend deltas vs the previous window, a real knowledge card, and a lazily-loaded, cached LLM welcome summary. ### Highlights - **Two contextual banners** on the overview: - **Inbox banner** — prompts the user to connect an inbox when the assistant has none, so it can actually do work. - **Coverage banner** — warns when FAQ coverage is below 85% with more than 100 responses pending review, linking straight to the pending queue. Dismissal persists per-assistant for 24h via localStorage. - **Batched stats builder** (`Captain::AssistantStatsBuilder`) computes both windows in single FILTER-aggregated scans to cut round trips, behind new `stats`/`summary` endpoints. - **Cards included but intentionally left dummy / not rendered yet:** `ResponseQualityCard` (flagged responses) and `CreditUsageCard` (credit usage + daily chart). Credits are an account-wide counter with no per-assistant or daily history, so there is no real data to back them yet; they ship in the codebase but are not wired into the page. ### Index migration - Replaces `index_messages_on_sender_type_and_sender_id` with `index_messages_on_sender_and_created` `(sender_type, sender_id, created_at)`. - **Why it helps:** the per-assistant windowed lookups filter `sender_*` *and* a `created_at` range. The old 2-column index matched every lifetime row for the assistant and filtered the time slice at the heap (~89% of rows discarded); adding `created_at` as a range column lets Postgres scan only the window, and fixes the row-count estimate so the planner picks a hash join over a nested loop on `reporting_events`. - **Why dropping the old index is safe:** the new index is a left-prefix superset `(sender_type, sender_id, ...)`, so every query the old one served is still served. No code references it by name, and dropping it keeps write amplification on `messages` neutral. Built/dropped with `CONCURRENTLY` and `if_not_exists`/`if_exists` guards. ## Preview <img width="2572" height="1754" alt="CleanShot 2026-06-29 at 22 38 51@2x" src="https://github.com/user-attachments/assets/3798d09e-7850-48e4-b2cd-508533f15cea" /> ## Banners #### Inbox connect alert <img width="2178" height="612" alt="CleanShot 2026-06-30 at 14 26 55@2x" src="https://github.com/user-attachments/assets/373c371c-bb7d-4291-a0f9-620673078302" /> #### Coverage alert <img width="2178" height="612" alt="CleanShot 2026-06-30 at 14 25 41@2x" src="https://github.com/user-attachments/assets/e12d6308-11b6-4ba2-88a2-8a3077dd3e8f" /> --------- Co-authored-by: Sivin Varghese <64252451+iamsivin@users.noreply.github.com>
333 lines
10 KiB
Ruby
333 lines
10 KiB
Ruby
# frozen_string_literal: true
|
|
|
|
# Reports Data Seeder
|
|
#
|
|
# Generates realistic test data for performance testing of reports and analytics.
|
|
# Creates conversations, messages, contacts, agents, teams, and labels with proper
|
|
# reporting events (first response times, resolution times, etc.) using time travel
|
|
# to generate historical data with realistic timestamps.
|
|
#
|
|
# Usage:
|
|
# ACCOUNT_ID=1 ENABLE_ACCOUNT_SEEDING=true bundle exec rake db:seed:reports_data
|
|
#
|
|
# This will create:
|
|
# - 1000 conversations with realistic message exchanges
|
|
# - 100 contacts with realistic profiles
|
|
# - 20 agents assigned to teams and inboxes
|
|
# - 5 teams with realistic distribution
|
|
# - 30 labels with random assignments
|
|
# - 3 inboxes with agent assignments
|
|
# - 1 Captain assistant bound to a single web inbox, with knowledge (FAQs + documents)
|
|
# and a variety of assistant-handled conversations (auto-resolved, handed off,
|
|
# handled with a human, resolved-then-reopened) for the assistant overview page
|
|
# - Realistic reporting events with historical timestamps
|
|
#
|
|
# Note: This seeder clears existing data for the account before seeding.
|
|
|
|
require 'faker'
|
|
require_relative 'conversation_creator'
|
|
require_relative 'message_creator'
|
|
require_relative 'assistant_conversation_creator'
|
|
|
|
# rubocop:disable Rails/Output, Metrics/ClassLength
|
|
class Seeders::Reports::ReportDataSeeder
|
|
include ActiveSupport::Testing::TimeHelpers
|
|
|
|
TOTAL_CONVERSATIONS = 1000
|
|
TOTAL_CONTACTS = 100
|
|
TOTAL_AGENTS = 20
|
|
TOTAL_TEAMS = 5
|
|
TOTAL_LABELS = 30
|
|
TOTAL_INBOXES = 3
|
|
MESSAGES_PER_CONVERSATION = 5
|
|
# Captain assistant conversations, split across the outcomes the overview page reports on.
|
|
TOTAL_ASSISTANT_CONVERSATIONS = 120
|
|
ASSISTANT_KNOWLEDGE_APPROVED = 14
|
|
ASSISTANT_KNOWLEDGE_PENDING = 6
|
|
ASSISTANT_DOCUMENTS = 4
|
|
START_DATE = 3.months.ago # rubocop:disable Rails/RelativeDateConstant
|
|
END_DATE = Time.current
|
|
|
|
def initialize(account:)
|
|
raise 'Account Seeding is not allowed.' unless ENV.fetch('ENABLE_ACCOUNT_SEEDING', !Rails.env.production?)
|
|
|
|
@account = account
|
|
@teams = []
|
|
@agents = []
|
|
@labels = []
|
|
@inboxes = []
|
|
@contacts = []
|
|
@assistant = nil
|
|
@assistant_inbox = nil
|
|
end
|
|
|
|
def perform!
|
|
puts "Starting reports data seeding for account: #{@account.name}"
|
|
|
|
# Clear existing data
|
|
clear_existing_data
|
|
|
|
create_teams
|
|
create_agents
|
|
create_labels
|
|
create_inboxes
|
|
create_contacts
|
|
create_assistant
|
|
|
|
create_conversations
|
|
create_assistant_conversations
|
|
|
|
puts "Completed reports data seeding for account: #{@account.name}"
|
|
end
|
|
|
|
private
|
|
|
|
def clear_existing_data
|
|
puts "Clearing existing data for account: #{@account.id}"
|
|
clear_assistant_data
|
|
@account.teams.destroy_all
|
|
@account.conversations.destroy_all
|
|
@account.labels.destroy_all
|
|
@account.inboxes.destroy_all
|
|
@account.contacts.destroy_all
|
|
@account.agents.destroy_all
|
|
@account.reporting_events.destroy_all
|
|
end
|
|
|
|
# Delete Captain records directly (assistant associations are destroy_async, which
|
|
# would leave rows around mid-reseed); order respects foreign keys.
|
|
def clear_assistant_data
|
|
assistant_ids = Captain::Assistant.for_account(@account.id).select(:id)
|
|
Captain::AssistantResponse.by_account(@account.id).delete_all
|
|
Captain::Document.for_account(@account.id).delete_all
|
|
CaptainInbox.where(captain_assistant_id: assistant_ids).delete_all
|
|
Captain::Assistant.for_account(@account.id).delete_all
|
|
end
|
|
|
|
def create_teams
|
|
TOTAL_TEAMS.times do |i|
|
|
team = @account.teams.create!(
|
|
name: "#{Faker::Company.industry} Team #{i + 1}"
|
|
)
|
|
@teams << team
|
|
print "\rCreating teams: #{i + 1}/#{TOTAL_TEAMS}"
|
|
end
|
|
|
|
print "\n"
|
|
end
|
|
|
|
def create_agents
|
|
TOTAL_AGENTS.times do |i|
|
|
user = create_single_agent(i)
|
|
assign_agent_to_teams(user)
|
|
@agents << user
|
|
print "\rCreating agents: #{i + 1}/#{TOTAL_AGENTS}"
|
|
end
|
|
|
|
print "\n"
|
|
end
|
|
|
|
def create_single_agent(index)
|
|
random_suffix = SecureRandom.hex(4)
|
|
user = User.create!(
|
|
name: Faker::Name.name,
|
|
email: "agent_#{index + 1}_#{random_suffix}@#{@account.domain || 'example.com'}",
|
|
password: 'Password1!.',
|
|
confirmed_at: Time.current
|
|
)
|
|
user.skip_confirmation!
|
|
user.save!
|
|
|
|
AccountUser.create!(
|
|
account_id: @account.id,
|
|
user_id: user.id,
|
|
role: :agent
|
|
)
|
|
|
|
user
|
|
end
|
|
|
|
def assign_agent_to_teams(user)
|
|
teams_to_assign = @teams.sample(rand(1..3))
|
|
teams_to_assign.each do |team|
|
|
TeamMember.create!(
|
|
team_id: team.id,
|
|
user_id: user.id
|
|
)
|
|
end
|
|
end
|
|
|
|
def create_labels
|
|
TOTAL_LABELS.times do |i|
|
|
label = @account.labels.create!(
|
|
title: "Label-#{i + 1}-#{Faker::Lorem.word}",
|
|
description: Faker::Company.catch_phrase,
|
|
color: Faker::Color.hex_color
|
|
)
|
|
@labels << label
|
|
print "\rCreating labels: #{i + 1}/#{TOTAL_LABELS}"
|
|
end
|
|
|
|
print "\n"
|
|
end
|
|
|
|
def create_inboxes
|
|
TOTAL_INBOXES.times do |_i|
|
|
inbox = create_single_inbox
|
|
assign_agents_to_inbox(inbox)
|
|
@inboxes << inbox
|
|
print "\rCreating inboxes: #{@inboxes.size}/#{TOTAL_INBOXES}"
|
|
end
|
|
|
|
print "\n"
|
|
end
|
|
|
|
def create_single_inbox
|
|
channel = Channel::WebWidget.create!(
|
|
website_url: "https://#{Faker::Internet.domain_name}",
|
|
account_id: @account.id
|
|
)
|
|
|
|
@account.inboxes.create!(
|
|
name: "#{Faker::Company.name} Website",
|
|
channel: channel
|
|
)
|
|
end
|
|
|
|
def assign_agents_to_inbox(inbox)
|
|
agents_to_assign = if @inboxes.empty?
|
|
# First inbox gets all agents to ensure coverage
|
|
@agents
|
|
else
|
|
# Subsequent inboxes get random selection with some overlap
|
|
min_agents = [@agents.size / TOTAL_INBOXES, 10].max
|
|
max_agents = [(@agents.size * 0.8).to_i, 50].min
|
|
@agents.sample(rand(min_agents..max_agents))
|
|
end
|
|
|
|
agents_to_assign.each do |agent|
|
|
InboxMember.create!(inbox: inbox, user: agent)
|
|
end
|
|
end
|
|
|
|
def create_contacts
|
|
TOTAL_CONTACTS.times do |i|
|
|
contact = @account.contacts.create!(
|
|
name: Faker::Name.name,
|
|
email: Faker::Internet.email,
|
|
phone_number: Faker::PhoneNumber.cell_phone_in_e164,
|
|
identifier: SecureRandom.uuid,
|
|
additional_attributes: {
|
|
company: Faker::Company.name,
|
|
city: Faker::Address.city,
|
|
country: Faker::Address.country,
|
|
customer_since: Faker::Date.between(from: 2.years.ago, to: Time.zone.today)
|
|
}
|
|
)
|
|
@contacts << contact
|
|
|
|
print "\rCreating contacts: #{i + 1}/#{TOTAL_CONTACTS}"
|
|
end
|
|
|
|
print "\n"
|
|
end
|
|
|
|
# One assistant, bound to a single web inbox (the first one), as the overview page expects.
|
|
def create_assistant
|
|
@account.enable_features!('captain_integration', 'captain_integration_v2')
|
|
@assistant_inbox = @inboxes.first
|
|
@assistant = Captain::Assistant.create!(
|
|
account: @account,
|
|
name: "#{Faker::Company.name} Copilot",
|
|
description: 'Captain assistant handling website support conversations.',
|
|
config: { feature_faq: true, feature_memory: true, product_name: @account.name }
|
|
)
|
|
CaptainInbox.create!(captain_assistant: @assistant, inbox: @assistant_inbox)
|
|
create_assistant_knowledge
|
|
|
|
puts "Created assistant '#{@assistant.name}' for inbox '#{@assistant_inbox.name}'"
|
|
end
|
|
|
|
def create_assistant_knowledge
|
|
ASSISTANT_KNOWLEDGE_APPROVED.times { create_assistant_response(:approved) }
|
|
ASSISTANT_KNOWLEDGE_PENDING.times { create_assistant_response(:pending) }
|
|
|
|
ASSISTANT_DOCUMENTS.times do
|
|
Captain::Document.create!(
|
|
account: @account,
|
|
assistant: @assistant,
|
|
name: Faker::Company.catch_phrase,
|
|
external_link: "https://#{Faker::Internet.domain_name}/#{Faker::Internet.slug}",
|
|
content: Faker::Lorem.paragraphs(number: rand(2..4)).join("\n\n"),
|
|
status: :available,
|
|
sync_status: :synced
|
|
)
|
|
end
|
|
end
|
|
|
|
def create_assistant_response(status)
|
|
Captain::AssistantResponse.create!(
|
|
account: @account,
|
|
assistant: @assistant,
|
|
question: "#{Faker::Lorem.sentence(word_count: rand(4..8)).chomp('.')}?",
|
|
answer: Faker::Lorem.paragraph(sentence_count: rand(2..4)),
|
|
status: status
|
|
)
|
|
end
|
|
|
|
def create_assistant_conversations
|
|
creator = Seeders::Reports::AssistantConversationCreator.new(
|
|
account: @account,
|
|
assistant: @assistant,
|
|
inbox: @assistant_inbox,
|
|
resources: { contacts: @contacts, agents: @agents }
|
|
)
|
|
|
|
outcomes = assistant_outcome_distribution
|
|
outcomes.each_with_index do |outcome, i|
|
|
created_at = Faker::Time.between(from: 65.days.ago, to: END_DATE)
|
|
creator.create_conversation(created_at: created_at, outcome: outcome)
|
|
|
|
print "\rCreating assistant conversations: #{i + 1}/#{outcomes.size}"
|
|
end
|
|
|
|
print "\n"
|
|
end
|
|
|
|
# Weighted mix of outcomes so every overview metric has meaningful numbers, shuffled
|
|
# so they interleave across the time span rather than clustering by type.
|
|
def assistant_outcome_distribution
|
|
counts = {
|
|
resolved_by_assistant: (TOTAL_ASSISTANT_CONVERSATIONS * 0.4).round,
|
|
handled_by_both: (TOTAL_ASSISTANT_CONVERSATIONS * 0.25).round,
|
|
handed_off: (TOTAL_ASSISTANT_CONVERSATIONS * 0.2).round,
|
|
resolved_and_reopened: (TOTAL_ASSISTANT_CONVERSATIONS * 0.15).round
|
|
}
|
|
counts.flat_map { |outcome, count| [outcome] * count }.shuffle
|
|
end
|
|
|
|
def create_conversations
|
|
conversation_creator = Seeders::Reports::ConversationCreator.new(
|
|
account: @account,
|
|
resources: {
|
|
contacts: @contacts,
|
|
inboxes: @inboxes,
|
|
teams: @teams,
|
|
labels: @labels,
|
|
agents: @agents
|
|
}
|
|
)
|
|
|
|
TOTAL_CONVERSATIONS.times do |i|
|
|
created_at = Faker::Time.between(from: START_DATE, to: END_DATE)
|
|
conversation_creator.create_conversation(created_at: created_at)
|
|
|
|
completion_percentage = ((i + 1).to_f / TOTAL_CONVERSATIONS * 100).round
|
|
print "\rCreating conversations: #{i + 1}/#{TOTAL_CONVERSATIONS} (#{completion_percentage}%)"
|
|
end
|
|
|
|
print "\n"
|
|
end
|
|
end
|
|
# rubocop:enable Rails/Output, Metrics/ClassLength
|