Tightens the resolved-conversation FAQ generator so it only proposes
durable, reusable FAQ candidates supported by human support-agent
messages. The implementation now sends a conversation-FAQ-specific
transcript to the LLM: customer messages plus real human support-agent
messages only, excluding bot, private, activity, and template messages.
## Closes
-
https://linear.app/chatwoot/issue/CW-7494/tighten-conversation-faq-generation-prompt
## What changed
- Added a human-only transcript builder in `ConversationFaqService`
instead of using the generic `conversation.to_llm_text` output.
- Excluded bot/agent-bot messages before the LLM call, which removes the
main bot-line leakage class deterministically.
- Preserved native-channel human replies where outgoing messages are
stored as `external_echo` without a `User` sender.
- Kept a prompt decision gate requiring each FAQ to be backed by a
complete public human-agent answer.
- Added generic no-FAQ classes for spam, wrong-service conversations,
private account/payment/order/certificate/troubleshooting cases, support
workflow mechanics, and direct-link/file/quote outputs.
- Added a separate `conversation_faq_generation` model route defaulting
to `gpt-5.2`, while keeping `document_faq_generation` on its existing
`gpt-4.1-mini` default. Conversation FAQ generation passes that feature
default ahead of the legacy global `CAPTAIN_OPEN_AI_MODEL` setting
unless an account-level override is configured.
- Kept the prompt domain-neutral so it can still generate reusable
product, service, policy, setup, and process FAQs outside SaaS contexts.
## Sampling notes
- Production Langfuse traces showed `llm.captain.conversation_faq` calls
using `gpt-4.1` in the sampled account set.
- Locally, `Llm::FeatureRouter.resolve(feature:
'conversation_faq_generation')` now resolves to `gpt-5.2`.
- Reviewed recent production `llm.captain.conversation_faq` traces
across 13+ accounts in compact form.
- Replayed 20 full traces across 10 accounts/domains, including
education, hosting, retail/auto, APIs, logistics, tax/fiscal workflows,
and Chatwoot account 1.
- Explicit `gpt-5.2` replay with human-only conversation history
returned no FAQ for 15/20 traces.
- A comparison replay with `gpt-4.1-mini` returned no FAQ for only 7/20
traces, bringing back several private/order/payment/support-workflow
cases.
- Remaining non-empty `gpt-5.2` outputs are now mostly
borderline/possibly useful human-agent-derived FAQs rather than obvious
bot-sourced answers.
## How to test
- Resolve conversations where the answer came only from the bot; no
pending FAQ should be generated.
- Resolve spam, unrelated, wrong-service, or private
payment/order/account conversations; no pending FAQ should be generated.
- Resolve conversations that require account/order/payment/login/private
verification or a human handoff; no pending FAQ should be generated.
- Resolve a conversation where a human agent gives a stable, reusable
help-center answer; the generated pending FAQ should be general and
self-contained.
# Pull Request Template
## Description
defaults new accounts to captain v2
## Type of change
- [x] New feature (non-breaking change which adds functionality)
## How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide
instructions so we can reproduce. Please also list any relevant details
for your test configuration.
locally and specs
## Checklist:
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] Any dependent changes have been merged and published in downstream
modules
## Description
Adds the foundation for feature-specific LLM model routing so Captain AI
features can resolve their effective provider/model from code defaults
and account-level overrides. This fixes the provider metadata key in
`config/llm.yml`, adds `Llm::FeatureRouter`, and routes existing
`CaptainFeaturable` model defaults through the shared resolver.
Fixes https://linear.app/chatwoot/issue/CW-7425/test-new-models
## Type of change
- [ ] Bug fix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality not to work as expected)
- [ ] This change requires a documentation update
## How Has This Been Tested?
- `bundle exec rspec spec/lib/llm/models_spec.rb
spec/lib/llm/feature_router_spec.rb
spec/models/concerns/captain_featurable_spec.rb` - 23 examples, 0
failures
- `bundle exec rubocop lib/llm/models.rb lib/llm/feature_router.rb
app/models/concerns/captain_featurable.rb spec/lib/llm/models_spec.rb
spec/lib/llm/feature_router_spec.rb
spec/models/concerns/captain_featurable_spec.rb` - no offenses
- `bundle exec ruby -e "require 'yaml'; config =
YAML.load_file('config/llm.yml'); abort('missing providers') unless
config['providers']; abort('missing models') unless config['models'];
abort('missing features') unless config['features']; puts 'llm.yml ok'"`
- `git diff --check`
## Checklist:
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [ ] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged and published in downstream
modules
