Compare commits

...
Author SHA1 Message Date
Shivam Mishra 8e97d25d35 feat: update rake task to include ARQ 2025-10-13 12:15:15 +05:30
Shivam Mishra 7317a083ea feat: add ARQ to scenario 2025-10-13 12:14:47 +05:30
Shivam Mishra 44fe854aab feat: include ARQ 2025-10-13 11:47:24 +05:30
8 changed files with 192 additions and 1 deletions
@@ -115,6 +115,10 @@ class Captain::Assistant < ApplicationRecord
}
end
def agent_response_schema
Captain::AssistantResponseSchema
end
def default_avatar_url
"#{ENV.fetch('FRONTEND_URL', nil)}/assets/images/dashboard/captain/logo.svg"
end
@@ -53,6 +53,10 @@ class Captain::Scenario < ApplicationRecord
}
end
def agent_response_schema
Captain::ScenarioResponseSchema
end
private
def agent_name
+1 -1
View File
@@ -47,7 +47,7 @@ module Concerns::Agentable
end
def agent_response_schema
Captain::ResponseSchema
raise NotImplementedError, "#{self.class} must define agent_response_schema"
end
def prompt_context
@@ -0,0 +1,20 @@
# frozen_string_literal: true
class Captain::AssistantResponseSchema < RubyLLM::Schema
string :response, description: 'The message to send to the user'
string :reasoning, description: "Agent's thought process tying the ARQ back to the final reply"
object :arq_plan, description: 'Attentive Reasoning Query plan used before producing the response' do
array :instructions_to_restate, of: :string,
description: 'Ordered guardrails, guidelines, tool duties, and routing expectations restated prior to reply'
array :response_compliance_checklist, of: :string,
description: 'Yes/no checklist confirming the response satisfies every critical obligation'
array :scenario_routing_candidates, of: :string,
description: 'Scenarios evaluated with quick notes on whether they should receive the handoff'
string :handoff_decision, description: 'Scenario key selected for handoff, or "self" when responding directly'
string :handoff_rationale, description: 'Why the chosen handoff path best serves the request while respecting constraints'
boolean :ready_to_respond,
description: 'Indicates all checklist items are satisfied and the assistant is cleared to draft the response'
string :notes, description: '≤20 word justification summarizing how obligations will be met or what is outstanding'
end
end
@@ -41,6 +41,65 @@ Always respect these boundaries:
{% endfor %}
{% endif -%}
# Attentive Reasoning Blueprint
Before composing the customer-facing `response`, fill the `arq_plan` object in your JSON output using the structure below. This ARQ keeps critical rules fresh and prevents guideline drift.
```json
{
"instructions_to_restate": [],
"response_compliance_checklist": [],
"scenario_routing_candidates": [],
"handoff_decision": "",
"handoff_rationale": "",
"ready_to_respond": true,
"notes": ""
}
```
- **`instructions_to_restate`** must list the non-negotiable directives you need top-of-mind **in this order**. Restate them as concrete sentences, not shorthand.
1. Guardrails that must never be broken.
2. Response guidelines that drive tone, content, or format.
3. Mandatory tool usage rules (e.g., FAQ lookup before citing product facts, handoff requirements).
4. Scenario routing expectations that influence ownership of this turn.
- **`response_compliance_checklist`** converts the above obligations into yes/no questions you can confirm before answering (e.g., "Did I verify every product statement via `captain--tools--faq_lookup`?").
- **`scenario_routing_candidates`** enumerates every scenario you actively considered in the format `"Scenario Name → reason it matches or not"`.
- **`handoff_decision`** is either the scenario key you will hand off to (e.g., `"billing_support"`) or `"self"` if you will answer directly.
- **`handoff_rationale`** briefly justifies why the chosen path best serves the customer and respects instructions.
- **`ready_to_respond`** stays `false` until every checklist item would be satisfied by your drafted reply.
- **`notes`** is ≤20 words summarizing how you will meet the checklist or what remains unresolved if `ready_to_respond` is `false`.
Use the live configuration below when populating the arrays:
{% if guardrails.size > 0 -%}
- Guardrails:
{% for guardrail in guardrails -%}
- {{ guardrail }}
{% endfor -%}
{% endif -%}
{% if response_guidelines.size > 0 -%}
- Response guidelines:
{% for guideline in response_guidelines -%}
- {{ guideline }}
{% endfor -%}
{% endif -%}
- Tool discipline reminders:
- Use `captain--tools--faq_lookup` before sharing product facts about {{product_name}}.
- Only call tools whose prerequisites are met and whose results will influence the reply.
- Summarize tool outputs so the customer can act on them.
- Scenario routing expectations:
- Treat routing as a primary responsibility: hand off when any scenario agent is better suited.
- Evaluate enabled scenario agents before handling the request yourself.
{% if scenarios.size > 0 -%}
{% for scenario in scenarios -%}
- {{ scenario.title }} → trigger with `handoff_to_{{ scenario.key }}` when {{ scenario.description | downcase }} applies.
{% endfor -%}
{% endif -%}
Once `ready_to_respond` would be `true`, draft the `response` and provide a concise, human-readable rationale in the `reasoning` field referencing how the checklist—including the handoff evaluation—was satisfied.
# Decision Framework
## 1. Analyze the Request
@@ -39,6 +39,62 @@ Always respect these boundaries:
{% endfor %}
{% endif -%}
# Attentive Reasoning Blueprint
Before replying, populate the `arq_plan` object in your JSON output using this structure to confirm scope alignment, tool usage, and potential return handoffs:
```json
{
"instructions_to_restate": [],
"response_compliance_checklist": [],
"scope_risks": [],
"tool_plan": [],
"handoff_back_decision": "",
"handoff_back_rationale": "",
"ready_to_respond": true,
"notes": ""
}
```
- **`instructions_to_restate`**: Concrete sentences covering guardrails, scenario instructions, and any required tool discipline.
- **`response_compliance_checklist`**: Yes/no checks confirming you honour the scenario-specific expectations and guardrails.
- **`scope_risks`**: Bulleted observations about user needs that might fall outside this scenario; include at least one entry even if it is "None identified".
- **`tool_plan`**: Sequence each tool you intend to call (or note `none`) with a justification.
- **`handoff_back_decision`**: Either `"handoff_to_{{ assistant_name }}"` when returning control or `"stay"` when you will respond directly.
- **`handoff_back_rationale`**: Brief explanation supporting the decision.
- **`ready_to_respond`**: Set to `false` until checklist items are satisfied and scope risks are resolved.
- **`notes`**: ≤20 words summarizing how you are meeting obligations or what remains open when not ready.
Use the information below when building the ARQ:
{% if response_guidelines.size > 0 -%}
- Response guidelines to honour:
{% for guideline in response_guidelines -%}
- {{ guideline }}
{% endfor -%}
{% endif -%}
{% if guardrails.size > 0 -%}
- Guardrails to restate:
{% for guardrail in guardrails -%}
- {{ guardrail }}
{% endfor -%}
{% endif -%}
- Scenario duties:
- Follow the scenario instructions verbatim; stay within scope.
- Return control via `handoff_to_{{ assistant_name }}` when user needs extend beyond this scenario.
- Tool discipline:
{% if tools.size > 0 -%}
{% for tool in tools -%}
- Use `{{ tool.id }}` when {{ tool.description | downcase }}.
{% endfor -%}
{% else -%}
- No dedicated tools available; rely on provided context.
{% endif -%}
Once `ready_to_respond` would be `true`, craft the customer-facing `response` and summarize how the checklist was satisfied in the `reasoning` field.
{% if tools.size > 0 -%}
# Available Tools
You have access to these tools:
@@ -0,0 +1,25 @@
# frozen_string_literal: true
class Captain::ScenarioResponseSchema < RubyLLM::Schema
string :response, description: 'Customer-facing message produced by the scenario agent'
string :reasoning, description: 'Short explanation connecting the ARQ plan to the final response or handoff'
object :arq_plan, description: 'Scenario Attentive Reasoning Query captured before responding' do
array :instructions_to_restate, of: :string,
description: 'Concrete guardrails, scenario instructions, and tool rules reiterated for this turn'
array :response_compliance_checklist, of: :string,
description: 'Yes/no questions verifying the response adheres to obligations'
array :scope_risks, of: :string,
description: 'Potential conversation needs outside this scenarios scope (or a “None identified” entry)'
array :tool_plan, of: :string,
description: 'Planned tool calls in order, each with a justification or “none”'
string :handoff_back_decision,
description: 'Either "handoff_to_<assistant_key>" to return control or "stay" to respond directly'
string :handoff_back_rationale,
description: 'Brief justification for staying or handing back to the assistant'
boolean :ready_to_respond,
description: 'True only when checklist is satisfied and scope risks are addressed'
string :notes,
description: '≤20 word summary of how obligations will be met or what remains open if not ready'
end
end
+23
View File
@@ -180,10 +180,12 @@ class CaptainChatSession
def display_response(result)
response_text = result['response'] || 'No response generated'
reasoning = result['reasoning']
arq_plan = result['arq_plan']
puts dim_text("\n#{@current_system_messages.join("\n")}") if @current_system_messages.any?
puts response_text
puts dim_italic_text("(Reasoning: #{reasoning})") if reasoning && reasoning != 'Processed by agent'
print_arq_plan(arq_plan)
add_to_history('assistant', response_text, reasoning: reasoning)
end
@@ -206,6 +208,27 @@ class CaptainChatSession
@message_history << message
end
def print_arq_plan(arq_plan)
return if arq_plan.blank?
puts dim_text('ARQ Plan:')
arq_plan.each do |key, value|
formatted_value = format_arq_value(value)
puts dim_text(" #{key}: #{formatted_value}")
end
end
def format_arq_value(value)
case value
when Array
value.empty? ? '[]' : value.map { |entry| "- #{entry}" }.join("\n ")
when Hash
value.transform_values { |nested| format_arq_value(nested) }
else
value.to_s
end
end
def clear_history
@message_history.clear
puts 'Message history cleared'