Tightens the resolved-conversation FAQ generator so it only proposes
durable, reusable FAQ candidates supported by human support-agent
messages. The implementation now sends a conversation-FAQ-specific
transcript to the LLM: customer messages plus real human support-agent
messages only, excluding bot, private, activity, and template messages.
## Closes
-
https://linear.app/chatwoot/issue/CW-7494/tighten-conversation-faq-generation-prompt
## What changed
- Added a human-only transcript builder in `ConversationFaqService`
instead of using the generic `conversation.to_llm_text` output.
- Excluded bot/agent-bot messages before the LLM call, which removes the
main bot-line leakage class deterministically.
- Preserved native-channel human replies where outgoing messages are
stored as `external_echo` without a `User` sender.
- Kept a prompt decision gate requiring each FAQ to be backed by a
complete public human-agent answer.
- Added generic no-FAQ classes for spam, wrong-service conversations,
private account/payment/order/certificate/troubleshooting cases, support
workflow mechanics, and direct-link/file/quote outputs.
- Added a separate `conversation_faq_generation` model route defaulting
to `gpt-5.2`, while keeping `document_faq_generation` on its existing
`gpt-4.1-mini` default. Conversation FAQ generation passes that feature
default ahead of the legacy global `CAPTAIN_OPEN_AI_MODEL` setting
unless an account-level override is configured.
- Kept the prompt domain-neutral so it can still generate reusable
product, service, policy, setup, and process FAQs outside SaaS contexts.
## Sampling notes
- Production Langfuse traces showed `llm.captain.conversation_faq` calls
using `gpt-4.1` in the sampled account set.
- Locally, `Llm::FeatureRouter.resolve(feature:
'conversation_faq_generation')` now resolves to `gpt-5.2`.
- Reviewed recent production `llm.captain.conversation_faq` traces
across 13+ accounts in compact form.
- Replayed 20 full traces across 10 accounts/domains, including
education, hosting, retail/auto, APIs, logistics, tax/fiscal workflows,
and Chatwoot account 1.
- Explicit `gpt-5.2` replay with human-only conversation history
returned no FAQ for 15/20 traces.
- A comparison replay with `gpt-4.1-mini` returned no FAQ for only 7/20
traces, bringing back several private/order/payment/support-workflow
cases.
- Remaining non-empty `gpt-5.2` outputs are now mostly
borderline/possibly useful human-agent-derived FAQs rather than obvious
bot-sourced answers.
## How to test
- Resolve conversations where the answer came only from the bot; no
pending FAQ should be generated.
- Resolve spam, unrelated, wrong-service, or private
payment/order/account conversations; no pending FAQ should be generated.
- Resolve conversations that require account/order/payment/login/private
verification or a human handoff; no pending FAQ should be generated.
- Resolve a conversation where a human agent gives a stable, reusable
help-center answer; the generated pending FAQ should be general and
self-contained.
