Commit Graph
2 Commits
Author SHA1 Message Date
Shivam MishraandGitHub d9c07fe2e9 feat: expose onboarding help center generation status (#14671)
This PR adds a reload-safe onboarding Help Center generation status
path. Previously, generation progress was pushed through ActionCable,
which meant the onboarding UI could lose context after a page reload or
missed websocket event. The new endpoint exposes the current generation
id, raw Redis generation state, and Help Center article/category counts
so clients can recover state by fetching from the backend.

**What changed**

- Added an onboarding Help Center generation status endpoint.
- Persisted `help_center_generation_id` when generation is enqueued.
- Removed Help Center generation ActionCable broadcasts completely.
- Kept generation progress in Redis as the backend source of truth.
- Kept Help Center generation out of the onboarding hot path; the
existing onboarding controller does not start generation yet.

**How to test**

1. Start Help Center generation for an account.
2. Fetch the onboarding generation status endpoint and verify it returns
the generation id, Redis state, and article/category counts.
3. Reload the onboarding UI or client state and fetch again to confirm
progress can be recovered without relying on websocket events.
4. Verify skipped/completed generation states are reflected from Redis.

## Related PRs

- https://github.com/chatwoot/chatwoot/pull/14569
- https://github.com/chatwoot/chatwoot/pull/14568
- https://github.com/chatwoot/chatwoot/pull/14567
- https://github.com/chatwoot/chatwoot/pull/14619
- https://github.com/chatwoot/chatwoot/pull/14565 (Primary onboarding
PR)
2026-06-10 16:24:37 +05:30
Shivam MishraandGitHub 3d20a7b049 feat: generate Help Center for Onboarding (#14370)
## Manually triggering help center generation

Open a Rails console (`bundle exec rails console`):

```ruby
account = Account.find(<ACCOUNT_ID>)
user    = account.users.first

# Optional: refresh brand info from the customer's website
domain = 'example.com'
result = WebsiteBrandingService.new("noreply@#{domain}").perform
account.update!(
  name: result[:title].presence || account.name,
  custom_attributes: account.custom_attributes.merge('website' => domain, 'brand_info' => result)
)

# Optional: wipe existing portals so a fresh one is created
account.portals.destroy_all

Onboarding::HelpCenterCreationService.new(account, user).perform
```

Sidekiq must be running — articles are written by
`Onboarding::HelpCenterArticleGenerationJob`. Avoid running on
production; generation calls the LLM provider.


### Generation flow (Happy Path) 

```mermaid
sequenceDiagram
    autonumber

    participant Kickoff as HelpCenterCreationService
    participant DB as DB
    participant GenJob as HelpCenterArticleGenerationJob
    participant Curator as HelpCenterCurator
    participant Firecrawl as Firecrawl
    participant CuratorLLM as Curation LLM
    participant Redis as Redis Progress
    participant WriterJob as HelpCenterArticleWriterJob
    participant Builder as HelpCenterArticleBuilder
    participant WriterLLM as Writer LLM
    participant Cable as ActionCable

    Kickoff->>DB: Create portal for account<br/>homepage_link=https://chatwoot.com
    Kickoff->>DB: Attach brand logo if available
    Kickoff->>GenJob: Enqueue generation job<br/>account_id, portal_id, user_id, generation_id

    GenJob->>Curator: Curate help center plan
    Curator->>Firecrawl: map https://chatwoot.com<br/>search: docs help support faq
    Firecrawl-->>Curator: Return discovered links
    Curator->>CuratorLLM: Select categories + article plans<br/>from discovered links only
    CuratorLLM-->>Curator: Return categories, articles, allowed_urls

    GenJob->>DB: Create portal categories
    GenJob->>GenJob: Stamp articles with category_id
    GenJob->>GenJob: Filter article URLs against allowed_urls
    GenJob->>GenJob: Drop articles with no category<br/>or no approved source URLs

    GenJob->>Redis: Start progress<br/>status=generating, total=N, finished=0

    loop For each approved article
      GenJob->>WriterJob: Enqueue writer job<br/>title, category_id, approved URLs
    end

    par Writer jobs run independently
      WriterJob->>Builder: Build article from approved URLs
      Builder->>Firecrawl: batch_scrape approved URLs
      Firecrawl-->>Builder: Return Markdown source pages
      Builder->>WriterLLM: Rewrite sources into one article
      WriterLLM-->>Builder: Return title, description, Markdown content
      Builder->>DB: Create draft portal article<br/>meta.source_urls
      WriterJob->>Redis: Increment finished count
      WriterJob->>Cable: Broadcast help_center.article_generated
    end

    WriterJob->>Redis: If finished >= total<br/>mark status=completed
    WriterJob->>Cable: Broadcast help_center.generation_completed
```

### Redis State Management

```mermaid
 stateDiagram-v2
    [*] --> active_pointer_set
    active_pointer_set --> generating: generation job creates valid plan
    active_pointer_set --> skipped: curation skipped/failed

    generating --> generating: each writer job increments finished
    generating --> completed: finished == total
    generating --> ignored_completion: generation_id superseded

    skipped --> [*]
    completed --> [*]
    ignored_completion --> [*]
```
2026-05-21 16:25:01 +05:30