feat: add ARQ to scenario

This commit is contained in:
Shivam Mishra
2025-10-13 12:14:47 +05:30
parent 44fe854aab
commit 7317a083ea
8 changed files with 116 additions and 8 deletions
@@ -115,6 +115,10 @@ class Captain::Assistant < ApplicationRecord
}
end
def agent_response_schema
Captain::AssistantResponseSchema
end
def default_avatar_url
"#{ENV.fetch('FRONTEND_URL', nil)}/assets/images/dashboard/captain/logo.svg"
end
@@ -53,6 +53,10 @@ class Captain::Scenario < ApplicationRecord
}
end
def agent_response_schema
Captain::ScenarioResponseSchema
end
private
def agent_name
+1 -1
View File
@@ -47,7 +47,7 @@ module Concerns::Agentable
end
def agent_response_schema
Captain::ResponseSchema
raise NotImplementedError, "#{self.class} must define agent_response_schema"
end
def prompt_context
@@ -0,0 +1,20 @@
# frozen_string_literal: true
class Captain::AssistantResponseSchema < RubyLLM::Schema
string :response, description: 'The message to send to the user'
string :reasoning, description: "Agent's thought process tying the ARQ back to the final reply"
object :arq_plan, description: 'Attentive Reasoning Query plan used before producing the response' do
array :instructions_to_restate, of: :string,
description: 'Ordered guardrails, guidelines, tool duties, and routing expectations restated prior to reply'
array :response_compliance_checklist, of: :string,
description: 'Yes/no checklist confirming the response satisfies every critical obligation'
array :scenario_routing_candidates, of: :string,
description: 'Scenarios evaluated with quick notes on whether they should receive the handoff'
string :handoff_decision, description: 'Scenario key selected for handoff, or "self" when responding directly'
string :handoff_rationale, description: 'Why the chosen handoff path best serves the request while respecting constraints'
boolean :ready_to_respond,
description: 'Indicates all checklist items are satisfied and the assistant is cleared to draft the response'
string :notes, description: '≤20 word justification summarizing how obligations will be met or what is outstanding'
end
end
@@ -48,6 +48,9 @@ Before composing the customer-facing `response`, fill the `arq_plan` object in y
{
"instructions_to_restate": [],
"response_compliance_checklist": [],
"scenario_routing_candidates": [],
"handoff_decision": "",
"handoff_rationale": "",
"ready_to_respond": true,
"notes": ""
}
@@ -59,6 +62,9 @@ Before composing the customer-facing `response`, fill the `arq_plan` object in y
3. Mandatory tool usage rules (e.g., FAQ lookup before citing product facts, handoff requirements).
4. Scenario routing expectations that influence ownership of this turn.
- **`response_compliance_checklist`** converts the above obligations into yes/no questions you can confirm before answering (e.g., "Did I verify every product statement via `captain--tools--faq_lookup`?").
- **`scenario_routing_candidates`** enumerates every scenario you actively considered in the format `"Scenario Name → reason it matches or not"`.
- **`handoff_decision`** is either the scenario key you will hand off to (e.g., `"billing_support"`) or `"self"` if you will answer directly.
- **`handoff_rationale`** briefly justifies why the chosen path best serves the customer and respects instructions.
- **`ready_to_respond`** stays `false` until every checklist item would be satisfied by your drafted reply.
- **`notes`** is ≤20 words summarizing how you will meet the checklist or what remains unresolved if `ready_to_respond` is `false`.
@@ -39,6 +39,62 @@ Always respect these boundaries:
{% endfor %}
{% endif -%}
# Attentive Reasoning Blueprint
Before replying, populate the `arq_plan` object in your JSON output using this structure to confirm scope alignment, tool usage, and potential return handoffs:
```json
{
"instructions_to_restate": [],
"response_compliance_checklist": [],
"scope_risks": [],
"tool_plan": [],
"handoff_back_decision": "",
"handoff_back_rationale": "",
"ready_to_respond": true,
"notes": ""
}
```
- **`instructions_to_restate`**: Concrete sentences covering guardrails, scenario instructions, and any required tool discipline.
- **`response_compliance_checklist`**: Yes/no checks confirming you honour the scenario-specific expectations and guardrails.
- **`scope_risks`**: Bulleted observations about user needs that might fall outside this scenario; include at least one entry even if it is "None identified".
- **`tool_plan`**: Sequence each tool you intend to call (or note `none`) with a justification.
- **`handoff_back_decision`**: Either `"handoff_to_{{ assistant_name }}"` when returning control or `"stay"` when you will respond directly.
- **`handoff_back_rationale`**: Brief explanation supporting the decision.
- **`ready_to_respond`**: Set to `false` until checklist items are satisfied and scope risks are resolved.
- **`notes`**: ≤20 words summarizing how you are meeting obligations or what remains open when not ready.
Use the information below when building the ARQ:
{% if response_guidelines.size > 0 -%}
- Response guidelines to honour:
{% for guideline in response_guidelines -%}
- {{ guideline }}
{% endfor -%}
{% endif -%}
{% if guardrails.size > 0 -%}
- Guardrails to restate:
{% for guardrail in guardrails -%}
- {{ guardrail }}
{% endfor -%}
{% endif -%}
- Scenario duties:
- Follow the scenario instructions verbatim; stay within scope.
- Return control via `handoff_to_{{ assistant_name }}` when user needs extend beyond this scenario.
- Tool discipline:
{% if tools.size > 0 -%}
{% for tool in tools -%}
- Use `{{ tool.id }}` when {{ tool.description | downcase }}.
{% endfor -%}
{% else -%}
- No dedicated tools available; rely on provided context.
{% endif -%}
Once `ready_to_respond` would be `true`, craft the customer-facing `response` and summarize how the checklist was satisfied in the `reasoning` field.
{% if tools.size > 0 -%}
# Available Tools
You have access to these tools:
@@ -3,11 +3,4 @@
class Captain::ResponseSchema < RubyLLM::Schema
string :response, description: 'The message to send to the user'
string :reasoning, description: "Agent's thought process"
object :arq_plan, description: 'Attentive Reasoning Query plan used to prepare the response' do
array :instructions_to_restate, of: :string,
description: 'Ordered list of concrete guardrails, guidelines, tool rules, and routing expectations to repeat before responding'
array :response_compliance_checklist, of: :string, description: 'Yes/no questions confirming the response satisfies each critical obligation'
boolean :ready_to_respond, description: 'Indicates whether the checklist is fully satisfied prior to finalizing the response'
string :notes, description: 'Short justification (≤20 words) explaining how the checklist is or will be satisfied'
end
end
@@ -0,0 +1,25 @@
# frozen_string_literal: true
class Captain::ScenarioResponseSchema < RubyLLM::Schema
string :response, description: 'Customer-facing message produced by the scenario agent'
string :reasoning, description: 'Short explanation connecting the ARQ plan to the final response or handoff'
object :arq_plan, description: 'Scenario Attentive Reasoning Query captured before responding' do
array :instructions_to_restate, of: :string,
description: 'Concrete guardrails, scenario instructions, and tool rules reiterated for this turn'
array :response_compliance_checklist, of: :string,
description: 'Yes/no questions verifying the response adheres to obligations'
array :scope_risks, of: :string,
description: 'Potential conversation needs outside this scenarios scope (or a “None identified” entry)'
array :tool_plan, of: :string,
description: 'Planned tool calls in order, each with a justification or “none”'
string :handoff_back_decision,
description: 'Either "handoff_to_<assistant_key>" to return control or "stay" to respond directly'
string :handoff_back_rationale,
description: 'Brief justification for staying or handing back to the assistant'
boolean :ready_to_respond,
description: 'True only when checklist is satisfied and scope risks are addressed'
string :notes,
description: '≤20 word summary of how obligations will be met or what remains open if not ready'
end
end