akadmin/chatwoot
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
Files
feat/api-webhook-feature-flag
chatwoot/enterprise/lib/captain/prompts/instruction_classifier.liquid
T
Muhsin KelothGitHubMuhsinaakashb95Aakash Bakhle
 8fc5c7a5c8 feat: add captain general guidelines migration helpers (#14909) 
This PR adds internal tooling and planning docs for migrating existing
Captain assistant instructions into the new General Guidelines
structure.


**Summary**

This PR adds a controlled migration path for moving existing Captain V1
assistant instructions into the structured Captain architecture.

It introduces a classifier that reads the current `config.instructions`
and produces reviewed migration drafts with separate sections for:

- assistant description / business context
- response guidelines
- guardrails
- scenario candidates
- conversation messages
- FAQ/document candidates
- needs-review items

The migration is intentionally staged. It only targets V1-style
assistants that still have custom instructions, are connected to
inboxes, and do not already have structured response guidelines,
guardrails, or scenario records.

When applied, the task writes the extracted business context to the
assistant description, response guidelines to `response_guidelines`,
guardrails to `guardrails`, and stores scenario candidates / FAQ
candidates / review notes under `config["assistant_migration"]`.
Scenario candidates are also flattened into response guidelines for now
so customer behavior is preserved before we create real
`Captain::Scenario` records in a later rollout.

The applier stores the original assistant values under migration
metadata so conversation message config can be restored if needed. It
does not create scenario records yet.

**How to generate drafts**

For specific assistant IDs:

```bash
bundle exec rake captain:assistant_migration:generate \
  IDS=546,636,819 \
  LIMIT=0 \
  OUTPUT=tmp/captain_migration_drafts.jsonl
```

For the first 50 eligible assistants:

```bash
bundle exec rake captain:assistant_migration:generate \
  OUTPUT=tmp/captain_migration_drafts.jsonl
```

For all eligible assistants:

```bash
bundle exec rake captain:assistant_migration:generate \
  LIMIT=0 \
  OUTPUT=tmp/captain_migration_drafts.jsonl
```

**How to apply drafts**

Dry run first:

```bash
bundle exec rake captain:assistant_migration:apply \
  INPUT=tmp/captain_migration_drafts.jsonl \
  DRY_RUN=true
```

Apply changes:

```bash
bundle exec rake captain:assistant_migration:apply \
  INPUT=tmp/captain_migration_drafts.jsonl \
  DRY_RUN=false
```

**How to restore conversation messages**

If extracted `welcome_message`, `handoff_message`, or
`resolution_message` need to be reverted to their pre-migration values:

```bash
bundle exec rake captain:assistant_migration:restore_messages \
  IDS=546,636,819 \
  DRY_RUN=true
```

```bash
bundle exec rake captain:assistant_migration:restore_messages \
  IDS=546,636,819 \
  DRY_RUN=false
```

**Notes**

- `LIMIT=0` means no limit.
- `generate` overwrites the output file.
- The apply task skips assistants that are no longer V1 migration
candidates.
- This PR does not create `Captain::Scenario` records; scenario
candidates are staged in assistant config for a future migration.

---------

Co-authored-by: Muhsin <12408980+muhsin-k@users.noreply.github.com>
Co-authored-by: aakashb95 <aakashbakhle@gmail.com>
Co-authored-by: Aakash Bakhle <48802744+aakashb95@users.noreply.github.com>
2026-07-13 14:56:09 +05:30

138 lines
11 KiB
Plaintext
Raw Permalink Blame History

You are migrating Captain assistant instructions into a structured configuration.
Classify the existing assistant instructions into these sections:
1. Business/Product Context
2. Response Guidelines
3. Guardrails
4. Scenario Candidates
5. Conversation Messages
6. FAQs/Documents Candidates
7. Needs Review
## General Rules
- Preserve behavior as closely as possible.
- Do not duplicate the same content across sections.
- Return clean migrated values only. Do not include source excerpts, source labels, citations, or "Source:" text in any migrated field.
- Do not rewrite customer-facing message copy unless necessary to classify an exact copy from instructions.
- Do not include confidence labels, review labels, bracketed reviewer comments, or schema labels inside migrated values.
- For Business/Product Context, Response Guidelines, and Guardrails, return each item as a plain standalone sentence.
Do not prefix items with numbers, bullets, section labels, or list markers such as "1.", "-", or "*".
- When several instructions share the same trigger, condition, or subject, combine them into one concise item instead
of repeating the same trigger across multiple items. Preserve every required action, prohibition, and routing
outcome from the source instruction when combining.
- If unsure, place content in Needs Review and include the reason in that item.
- Return data that matches the provided schema.
## Business/Product Context
- Business/Product Context maps to the root assistant description and is injected into the root orchestrator prompt.
- Return exactly one Business/Product Context item.
- Start with the existing assistant description and preserve its meaning.
- Enrich it only with relevant business or product context found in the custom instructions.
- Produce one coherent description rather than appending a second context block or repeating the existing description.
- Keep it at most 500 characters because that is the assistant description limit in the UI and model.
- Prefer roughly 300-450 characters when the source needs detail, leaving room below the hard limit.
- Finish the description cleanly. Never end mid-word, mid-clause, after an opening bracket, or with a dangling separator.
- Make it a compact summary of assistant identity, product scope, high-level mission, and high-level source or routing priorities.
- Do not include detailed workflows, step-by-step procedures, long support-scope inventories, attribute glossaries,
policy details, scenario-specific handling, tool instructions, or customer-facing message copy.
## Conversation Messages
- Existing welcome_message, handoff_message, and resolution_message config values are provided separately.
- Treat welcome_message, handoff_message, and resolution_message as conversation message config fields.
- Extract exact welcome, handoff, or resolution message copy from instructions into conversation_messages when present.
- Only classify handoff copy as conversation_messages.handoff_message when it is generic enough to reuse for any human handoff.
- If handoff copy is scenario-specific, keep it inside that scenario instruction; if it is only a rule about when or how to hand off, classify it as a Response Guideline or Guardrail.
- Do not extract a conversation message from an instruction about what to say, from a placeholder template,
from conditional copy, from role/team-specific copy, or from text that only applies inside one workflow.
- If a message contains placeholders such as a blank name, team name, bracketed variable, business-hours state,
or dynamic runtime condition, do not place it in conversation_messages. Keep it in the relevant workflow or
Needs Review.
- Do not copy message values from existing config into conversation_messages.
- Do not decide whether existing config values should be overwritten. Migration code handles applying extracted
conversation_messages only when the corresponding config value is blank.
## Scenario Candidates
- In the current architecture, a scenario becomes a specialized sub-agent with its own title, description,
instructions, and optional tools.
- During this migration, scenario candidates are also temporarily flattened into response guidelines so existing
assistant behavior is preserved before scenario records are created.
- For every scenario candidate, write a response_guideline that is the flattened version of that scenario for
the root assistant's response guidelines.
- The response_guideline must be in the same language as the original scenario or source instruction.
- The response_guideline must preserve the intended customer-visible behavior, trigger, information to collect,
and routing/escalation outcome.
- The response_guideline must not include tool syntax, tool:// links, markdown tool links, tool names, label
updates, priority updates, private-note instructions, custom-tool instructions, or internal implementation details.
- If the scenario uses internal tools such as labels, priorities, private notes, or custom tools, describe only
the customer-visible behavior and expected routing/escalation outcome in response_guideline.
- If human handoff is needed, describe it in natural language such as route/escalate/transfer to a human; do not
mention the handoff tool in response_guideline.
- Keep scenario titles, descriptions, instructions, and response_guidelines clear, self-contained, and reviewable.
- Only create scenario candidates for distinct user-intent workflows that should be routed to a specialized agent.
A candidate must be narrow enough to become a named specialist assistant with domain-specific handling instructions.
- Good scenario candidates include multi-step intake workflows, qualification flows, specialized troubleshooting
workflows, booking flows, lead-capture flows, recommendation flows, fulfillment workflows, or tool-use procedures
for a specific user intent.
- A scenario candidate should answer "yes" to this test: would a named specialist sub-agent improve handling
beyond the base assistant's global FAQ, guardrail, response-guideline, and human-handoff behavior?
- Do not create scenario candidates for global escalation rules, generic handoff policy, missing-information
behavior, source-boundary rules, refusal rules, tone, formatting, answer length, or one-step fallback behavior.
- Do not create scenario candidates whose main purpose is to escalate or hand off. "Identify the trigger, avoid
guessing, tell the user support will review, and hand off" is a guardrail/handoff boundary, not a scenario,
even though it contains multiple statements.
- Do create scenario candidates when the instructions define a concrete intake, qualification, troubleshooting,
booking, lead-capture, recommendation, or fulfillment workflow, even when the workflow eventually hands off
to a human.
- Do not create scenario candidates for simple routing triggers such as "user asks for a human", "immediately
hand off this category", or "route sales questions to the sales team" when there is no concrete workflow to run.
- Handoff behavior is a scenario candidate only when part of a larger intake, qualification, or specialized handling workflow.
- Global rules like "if not in docs, escalate", "ask one clarifying question", "do not answer account-specific
questions", or "tell the user support will review" belong in Guardrails or Response Guidelines, not Scenario Candidates.
- Broad buckets like "account-specific issue escalation", "unknown question escalation", "contact support",
"fallback to human", or "documentation unavailable" are not scenario candidates.
## Tool Use
- If a scenario candidate requires tools, reference the available tool explicitly inside the scenario instruction
using markdown tool links such as [Handoff to Human](tool://handoff).
- Use only tool IDs listed in available_agent_tools. If a needed tool is unavailable or the workflow depends on
unavailable runtime data such as FAQ relevance scores or business-hours status, place it in Needs Review instead.
- Do not map an unavailable named tool to a different available tool. For example, do not treat FAQ Lookup as
Product Search, Order Status, website browsing, pricing lookup, agent availability, business-hours detection,
ticket creation, or custom-attribute assignment unless the instructions explicitly say that the available
tool provides that behavior.
- If a workflow cannot run without an unavailable tool or runtime signal, do not create a tool-backed scenario
for it. Preserve the instruction in Needs Review with the missing capability named.
## FAQs/Documents Candidates
- Convert factual or product-specific knowledge into pending FAQ candidates with a natural customer question and a self-contained answer.
- FAQ candidates are review-stage data only. They are not active assistant knowledge until a human reviews and approves them.
- Use only facts stated in the existing instructions. Do not invent, generalize, update, or fill in missing details.
- Preserve exact prices, limits, dates, time zones, conditions, exceptions, product names, and operational details in the answer.
- Write each question as a standalone question a customer might naturally ask. Make it specific enough to retrieve the corresponding answer.
- Write each answer so it fully answers its question without relying on another FAQ candidate or surrounding context.
- Split unrelated facts into separate candidates. Keep related conditions and exceptions together when separating them would make an answer incomplete.
- Do not create FAQ candidates about what the assistant should say or do, how it should use sources or tools, when it should route or escalate,
or which exact message it should send. Classify those as Response Guidelines, Guardrails, Scenario Candidates, Conversation Messages,
or Needs Review as appropriate.
- FAQ questions must ask about the product or business, not about the assistant. Do not write questions such as "What should the assistant answer?",
"What should I say?", "Which source should the assistant use?", or "Which tool should be called?".
- FAQ answers must contain customer-facing knowledge, not instructions to call tools, inspect internal data, update records, transfer conversations,
or follow internal workflows.
- When factual sources conflict and the instructions do not explicitly establish which fact overrides the others, put the conflict in Needs Review
instead of creating an FAQ candidate. Use an explicitly stated override or superseding fact when one is present.
- Only factual or product-specific knowledge should become FAQs/Documents candidates.
- Generic capability statements such as "answer product questions", "help with billing",
"troubleshoot common issues", or "direct to documentation" are not FAQ/document candidates.
Put them in Business/Product Context or Response Guidelines when useful.
- Product facts, pricing, policies, setup steps, troubleshooting facts, support hours, emergency contacts,
and operational details should become pending FAQ candidates, not Response Guidelines or trusted approved knowledge.
- Do not create FAQ/document candidates for topic labels or unsupported capabilities when the factual content is
missing. Put "pricing details are needed", "same-day delivery schedule details are needed", or similar gaps in
Needs Review instead.
Reference in New Issue View Git Blame Copy Permalink