Shivam MishraandGitHub
3d20a7b049
## Manually triggering help center generation
Open a Rails console (`bundle exec rails console`):
```ruby
account = Account.find(<ACCOUNT_ID>)
user = account.users.first
# Optional: refresh brand info from the customer's website
domain = 'example.com'
result = WebsiteBrandingService.new("noreply@#{domain}").perform
account.update!(
name: result[:title].presence || account.name,
custom_attributes: account.custom_attributes.merge('website' => domain, 'brand_info' => result)
)
# Optional: wipe existing portals so a fresh one is created
account.portals.destroy_all
Onboarding::HelpCenterCreationService.new(account, user).perform
```
Sidekiq must be running — articles are written by
`Onboarding::HelpCenterArticleGenerationJob`. Avoid running on
production; generation calls the LLM provider.
### Generation flow (Happy Path)
```mermaid
sequenceDiagram
autonumber
participant Kickoff as HelpCenterCreationService
participant DB as DB
participant GenJob as HelpCenterArticleGenerationJob
participant Curator as HelpCenterCurator
participant Firecrawl as Firecrawl
participant CuratorLLM as Curation LLM
participant Redis as Redis Progress
participant WriterJob as HelpCenterArticleWriterJob
participant Builder as HelpCenterArticleBuilder
participant WriterLLM as Writer LLM
participant Cable as ActionCable
Kickoff->>DB: Create portal for account<br/>homepage_link=https://chatwoot.com
Kickoff->>DB: Attach brand logo if available
Kickoff->>GenJob: Enqueue generation job<br/>account_id, portal_id, user_id, generation_id
GenJob->>Curator: Curate help center plan
Curator->>Firecrawl: map https://chatwoot.com<br/>search: docs help support faq
Firecrawl-->>Curator: Return discovered links
Curator->>CuratorLLM: Select categories + article plans<br/>from discovered links only
CuratorLLM-->>Curator: Return categories, articles, allowed_urls
GenJob->>DB: Create portal categories
GenJob->>GenJob: Stamp articles with category_id
GenJob->>GenJob: Filter article URLs against allowed_urls
GenJob->>GenJob: Drop articles with no category<br/>or no approved source URLs
GenJob->>Redis: Start progress<br/>status=generating, total=N, finished=0
loop For each approved article
GenJob->>WriterJob: Enqueue writer job<br/>title, category_id, approved URLs
end
par Writer jobs run independently
WriterJob->>Builder: Build article from approved URLs
Builder->>Firecrawl: batch_scrape approved URLs
Firecrawl-->>Builder: Return Markdown source pages
Builder->>WriterLLM: Rewrite sources into one article
WriterLLM-->>Builder: Return title, description, Markdown content
Builder->>DB: Create draft portal article<br/>meta.source_urls
WriterJob->>Redis: Increment finished count
WriterJob->>Cable: Broadcast help_center.article_generated
end
WriterJob->>Redis: If finished >= total<br/>mark status=completed
WriterJob->>Cable: Broadcast help_center.generation_completed
```
### Redis State Management
```mermaid
stateDiagram-v2
[*] --> active_pointer_set
active_pointer_set --> generating: generation job creates valid plan
active_pointer_set --> skipped: curation skipped/failed
generating --> generating: each writer job increments finished
generating --> completed: finished == total
generating --> ignored_completion: generation_id superseded
skipped --> [*]
completed --> [*]
ignored_completion --> [*]
```
66 lines
1.7 KiB
Ruby
66 lines
1.7 KiB
Ruby
class Onboarding::HelpCenterCurator
|
|
MAP_LIMIT = 500
|
|
MAP_SEARCH = 'docs help support faq'.freeze
|
|
MIN_ARTICLES = 3
|
|
|
|
Skipped = Onboarding::HelpCenterErrors::CurationSkipped
|
|
|
|
def initialize(account:)
|
|
@account = account
|
|
end
|
|
|
|
def perform
|
|
raise Skipped, 'Firecrawl not configured' unless Firecrawl::Configuration.configured?
|
|
raise Skipped, 'no website url' if website_url.blank?
|
|
|
|
links = discover_links
|
|
raise Skipped, 'map returned no links' if links.empty?
|
|
|
|
plan = curate(links)
|
|
raise Skipped, "only #{plan[:articles].size} articles curated (< #{MIN_ARTICLES} threshold)" if plan[:articles].size < MIN_ARTICLES
|
|
|
|
plan.merge(allowed_urls: extract_urls(links)).deep_stringify_keys
|
|
end
|
|
|
|
private
|
|
|
|
def discover_links
|
|
data = Firecrawl::Configuration.client.map(
|
|
website_url,
|
|
Firecrawl::Models::MapOptions.new(limit: MAP_LIMIT, search: MAP_SEARCH)
|
|
)
|
|
Array(data.links)
|
|
end
|
|
|
|
def extract_urls(links)
|
|
Array(links).filter_map do |link|
|
|
link['url'].presence
|
|
end.uniq
|
|
end
|
|
|
|
def curate(links)
|
|
response = Captain::Llm::HelpCenterCurationService.new(account: @account, links: links).perform
|
|
raise Skipped, "curator LLM error: #{response[:error]}" if response[:error]
|
|
|
|
response[:message] || { categories: [], articles: [] }
|
|
end
|
|
|
|
def website_url
|
|
@website_url ||= with_scheme(custom_attributes_website.presence || brand_info[:domain].presence)
|
|
end
|
|
|
|
def with_scheme(raw)
|
|
return raw if raw.blank?
|
|
|
|
raw.match?(%r{\Ahttps?://}i) ? raw : "https://#{raw}"
|
|
end
|
|
|
|
def custom_attributes_website
|
|
@account.custom_attributes['website']
|
|
end
|
|
|
|
def brand_info
|
|
@brand_info ||= (@account.custom_attributes['brand_info'] || {}).deep_symbolize_keys
|
|
end
|
|
end
|