akadmin/chatwoot
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

chore: update docs

Browse Source
This commit is contained in:
Muhsin
2026-02-28 17:54:29 +04:00
parent db96d0a34c
commit c3606d22ea
2 changed files with 235 additions and 155 deletions
Download Patch File Download Diff File Expand all files Collapse all files
captain-shopify-gap-competitor-plan.md
+128
View File
@@ -0,0 +1,128 @@
# Captain + Shopify Gap Analysis and Competitor Comparison Plan
## 1. Purpose
Create a clear plan to:
1. Identify product and technical gaps in current Captain + Shopify experience.
2. Compare current behavior with leading support/commerce competitors.
3. Convert findings into a prioritized implementation roadmap.
## 2. Current Baseline (Chatwoot)
Current implemented scope:
1. Shopify product search via Captain tool.
2. Shopify order lookup by contact email/phone.
3. Captain V2 + Shopify connection gating.
4. Fallback identity extraction from conversation text/history.
Out of scope in current version:
1. Order lookup by order ID.
2. Checkout/cart workflows.
3. Deep post-purchase workflows (returns/exchanges automation).
4. GraphQL-first Shopify data layer.
## 3. Competitor Set
Primary competitors for comparison:
1. Gorgias (Shopify-first support workflows).
2. Zendesk + Shopify integration.
3. Intercom (Fin + commerce integrations).
4. Tidio / similar SMB Shopify-focused AI support stacks.
Note: This plan focuses on capability and UX patterns, not pricing.
## 4. Comparison Dimensions
Evaluate each competitor and Chatwoot against the same rubric:
1. Identity resolution robustness (email/phone/order context).
2. Order lookup depth (status, fulfillment, tracking, edits).
3. Product intelligence quality (search relevance, detail depth).
4. Agent handoff quality after failed AI attempts.
5. Conversation memory and context carry-forward.
6. Actionability (read-only vs operational actions).
7. Error transparency and recovery UX.
8. Admin visibility (logs, traceability, debugging).
9. Setup friction (scopes, auth, reauthorization handling).
10. Reliability under ambiguous user messages.
## 5. Gap Assessment Method
### Step 1: Baseline test scenarios (Chatwoot)
Run a fixed scenario pack:
1. Product availability questions.
2. Specific product detail follow-up.
3. Order status lookup with:
- email present in contact
- email only in user message
- confirmation-only follow-up (`Yes`)
- phone-only identity
4. No-result and provider-error paths.
### Step 2: Competitor behavior capture
For each scenario:
1. Capture response quality.
2. Capture number of turns to resolution.
3. Capture failure behavior and escalation quality.
### Step 3: Scorecard
Use 0-3 scoring per dimension:
1. `0` = missing
2. `1` = weak/manual
3. `2` = functional
4. `3` = strong/best-in-class
## 6. Initial Gap Hypotheses
Likely current gaps to validate:
1. Order retrieval flexibility:
- No order-id lookup path yet.
2. Product detail richness:
- Limited details when Shopify product metadata is sparse.
3. Proactive recovery:
- AI asks for identity but does not persist verified identity for next turns.
4. Action workflows:
- No built-in operational actions (refund/cancel/update address/escalate with structured payload).
5. Observability UX:
- Logs exist, but no dedicated operational dashboard for Shopify tool outcomes.
## 7. Roadmap Framework (Post-Analysis)
### Phase A (Quick Wins)
1. Improve prompt/tool guidance for identity confirmation.
2. Add operator-facing diagnostics for identity source and lookup failure reason.
3. Expand product response template quality.
### Phase B (Core Capability)
1. Add optional order lookup by order ID (feature-flagged).
2. Persist verified contact identity with safeguards.
3. Add richer order timeline output (payment, fulfillment, tracking where available).
### Phase C (Strategic)
1. Move Shopify reads to GraphQL Admin API.
2. Add structured post-purchase workflows.
3. Add KPI dashboard for resolution rate and lookup success rate.
## 8. Success Metrics
Track before/after:
1. First-response resolution rate for order status requests.
2. Median turns to order lookup completion.
3. Percentage of order lookups failing with missing identifier.
4. Product query relevance satisfaction (manual QA scoring).
5. Human handoff rate for Shopify intents.
## 9. Deliverables
1. Gap matrix (Chatwoot vs each competitor by dimension).
2. Ranked backlog of gaps with impact and effort.
3. Implementation proposal with phased milestones.
4. QA plan for each planned feature.
## 10. Execution Checklist
1. Finalize scenario pack.
2. Run Chatwoot baseline and record outputs.
3. Run competitor comparisons and record outputs.
4. Complete scoring matrix.
5. Convert top gaps into engineering tickets.
6. Validate each implemented item against the same scenario pack.
shopify_advanced.md
+107 -155
View File
@@ -1,199 +1,151 @@
# `shopify_advanced.md` + Implementation Runbook (Enterprise, Captain V2, Products+Orders)
# Shopify Advanced (Implementation-Aligned Runbook)
## Summary
Implement advanced Shopify support for Captain as **Enterprise + Captain V2 only**, with **phase 1 limited to product search and order lookup**.
No Captain V1 work, no cart/checkout work in this phase.
This document reflects the implemented Shopify support for Captain in Chatwoot:
1. Enterprise-only
2. Captain V2-only (`captain_integration_v2`)
3. Phase 1 scope: product search + order lookup
4. Order lookup is by contact identity (`email` / `phone_number`) only
## Deliverables
1. Root doc file: `shopify_advanced.md` (this plan content).
2. Backend Shopify service layer (shared, read-only).
3. Two new Captain built-in tools:
- `shopify_search_products`
- `shopify_get_orders`
4. Tool exposure/runtime gating to Captain V2 + connected Shopify only.
5. Scope update: add `read_products`.
6. Specs for services, tools, and gating.
No Captain V1 behavior and no checkout/cart features are included.
## Important Interface Changes
1. New Captain tool IDs:
- `shopify_search_products`
- `shopify_get_orders`
2. Shopify required scopes update:
- add `read_products` to `REQUIRED_SCOPES`.
3. No new public REST endpoints for Captain.
## Implemented Tool Surface
1. `shopify_search_products`
2. `shopify_get_orders`
## Files to Create
Both tools are registered in:
[`/Users/muhsink/Documents/chatwoot/config/agents/tools.yml`](/Users/muhsink/Documents/chatwoot/config/agents/tools.yml)
### 1) Shared Shopify services
1. `app/services/integrations/shopify/client_service.rb`
2. `app/services/integrations/shopify/products_service.rb`
3. `app/services/integrations/shopify/orders_service.rb`
## Shopify Scope Requirements
1. Integration helper includes:
- `read_customers`
- `read_orders`
- `read_fulfillments`
- `read_products`
2. `read_products` is required for product search behavior.
### 2) Enterprise Captain tools
1. `enterprise/lib/captain/tools/shopify_base_tool.rb`
2. `enterprise/lib/captain/tools/shopify_search_products_tool.rb`
3. `enterprise/lib/captain/tools/shopify_get_orders_tool.rb`
Reference:
[`/Users/muhsink/Documents/chatwoot/app/helpers/shopify/integration_helper.rb`](/Users/muhsink/Documents/chatwoot/app/helpers/shopify/integration_helper.rb)
### 3) Specs
1. `spec/services/integrations/shopify/products_service_spec.rb`
2. `spec/services/integrations/shopify/orders_service_spec.rb`
3. `spec/enterprise/lib/captain/tools/shopify_base_tool_spec.rb`
4. `spec/enterprise/lib/captain/tools/shopify_search_products_tool_spec.rb`
5. `spec/enterprise/lib/captain/tools/shopify_get_orders_tool_spec.rb`
## Core Files
### Services
1. [`/Users/muhsink/Documents/chatwoot/app/services/integrations/shopify/client_service.rb`](/Users/muhsink/Documents/chatwoot/app/services/integrations/shopify/client_service.rb)
2. [`/Users/muhsink/Documents/chatwoot/app/services/integrations/shopify/products_service.rb`](/Users/muhsink/Documents/chatwoot/app/services/integrations/shopify/products_service.rb)
3. [`/Users/muhsink/Documents/chatwoot/app/services/integrations/shopify/orders_service.rb`](/Users/muhsink/Documents/chatwoot/app/services/integrations/shopify/orders_service.rb)
## Files to Modify
### Captain tools
1. [`/Users/muhsink/Documents/chatwoot/enterprise/lib/captain/tools/shopify_base_tool.rb`](/Users/muhsink/Documents/chatwoot/enterprise/lib/captain/tools/shopify_base_tool.rb)
2. [`/Users/muhsink/Documents/chatwoot/enterprise/lib/captain/tools/shopify_search_products_tool.rb`](/Users/muhsink/Documents/chatwoot/enterprise/lib/captain/tools/shopify_search_products_tool.rb)
3. [`/Users/muhsink/Documents/chatwoot/enterprise/lib/captain/tools/shopify_get_orders_tool.rb`](/Users/muhsink/Documents/chatwoot/enterprise/lib/captain/tools/shopify_get_orders_tool.rb)
1. [`/Users/muhsink/Documents/chatwoot/app/helpers/shopify/integration_helper.rb`](/Users/muhsink/Documents/chatwoot/app/helpers/shopify/integration_helper.rb)
- Add `read_products` to `REQUIRED_SCOPES`.
### Gating + prompt
1. [`/Users/muhsink/Documents/chatwoot/enterprise/app/models/captain/assistant.rb`](/Users/muhsink/Documents/chatwoot/enterprise/app/models/captain/assistant.rb)
2. [`/Users/muhsink/Documents/chatwoot/enterprise/lib/captain/prompts/assistant.liquid`](/Users/muhsink/Documents/chatwoot/enterprise/lib/captain/prompts/assistant.liquid)
2. [`/Users/muhsink/Documents/chatwoot/config/agents/tools.yml`](/Users/muhsink/Documents/chatwoot/config/agents/tools.yml)
- Register the 2 new tools with title/description/icon.
3. [`/Users/muhsink/Documents/chatwoot/enterprise/app/models/captain/assistant.rb`](/Users/muhsink/Documents/chatwoot/enterprise/app/models/captain/assistant.rb)
- Add helper `shopify_tools_enabled_for_v2?`.
- Auto-append Shopify tools in `agent_tools` when eligible.
- Filter tool metadata in `available_agent_tools` unless eligible.
4. [`/Users/muhsink/Documents/chatwoot/enterprise/lib/captain/prompts/assistant.liquid`](/Users/muhsink/Documents/chatwoot/enterprise/lib/captain/prompts/assistant.liquid)
- Add short instruction: use Shopify tools for product/order queries before FAQ fallback.
5. Optional refactor only (no contract change):
- [`/Users/muhsink/Documents/chatwoot/app/controllers/api/v1/accounts/integrations/shopify_controller.rb`](/Users/muhsink/Documents/chatwoot/app/controllers/api/v1/accounts/integrations/shopify_controller.rb)
- Reuse `OrdersService` internally for consistency.
## Implementation Details
### A. `ClientService`
Responsibilities:
1. Load connected hook (`app_id: 'shopify'`, status enabled).
2. Build Shopify REST client from hook token.
3. Parse granted scopes from hook settings.
4. Return normalized failure objects for:
## Service Behavior
### `Integrations::Shopify::ClientService`
1. Loads enabled Shopify hook (`app_id: 'shopify'`).
2. Builds Shopify REST admin client.
3. Parses granted scopes from hook settings.
4. Returns normalized results:
- `:not_connected`
- `:provider_error`
5. Emits structured Rails logs for connection and scope context.
### B. `ProductsService`
### `Integrations::Shopify::ProductsService`
Method:
- `search_products(query:, limit: 10)`
Behavior:
1. Validate connection + `read_products`.
2. Fetch active products by title query.
3. Normalize output:
1. Rejects blank query with `:no_results`.
2. Validates connection and `read_products` scope.
3. Primary search: active products by title.
4. Fallback search: fetch active products with larger limit, then in-memory keyword match against title/vendor/type.
5. Normalizes product payload:
- `id`, `title`, `vendor`, `product_type`, `handle`, `storefront_url`
- first variant `price`
- availability summary from variants.
4. Return normalized result/error:
- `:insufficient_scope`, `:no_results`, `:provider_error`.
- computed `availability` summary
6. Error mapping:
- `:insufficient_scope`, `:no_results`, `:provider_error`
### C. `OrdersService`
### `Integrations::Shopify::OrdersService`
Method:
- `orders_for_contact(email:, phone_number:, limit: 10)`
Behavior:
1. Require at least one identifier (`email` or `phone_number`).
2. Validate connection + scopes `read_customers` and `read_orders`.
3. Search customer by `email OR phone`.
4. Fetch orders for first matched customer.
5. Normalize output:
1. Requires at least one identifier (`email` or `phone_number`).
2. Validates connection and scopes `read_customers` + `read_orders`.
3. Runs customer search using `email OR phone`.
4. Fetches orders for first matching customer.
5. Normalizes order payload:
- `id`, `name`, `created_at`, `total_price`, `currency`
- `financial_status`, `fulfillment_status`
- `line_items` (capped)
- `admin_url`.
6. Return normalized result/error:
- `:missing_identifier`, `:no_results`, `:provider_error`.
- `admin_url`
6. Error mapping:
- `:missing_identifier`, `:insufficient_scope`, `:no_results`, `:provider_error`
### D. `ShopifyBaseTool`
Common behavior:
1. Extend `BasePublicTool`.
2. `active?` true only when:
- `captain_integration_v2` enabled
- Shopify hook enabled for account.
3. Common helpers:
- `v2_enabled?`
- `shopify_connected?`
- `resolve_contact_identity(tool_context.state)`
- scope guard + standardized reconnect message.
4. Tool-safe deterministic messages, no exceptions leaked.
## Tool Behavior
### Base tool (`Captain::Tools::ShopifyBaseTool`)
1. `active?` requires:
- account feature flag `captain_integration_v2`
- enabled Shopify integration hook
2. Common deterministic domain error formatting.
3. Identity resolution order for order lookup:
- explicit tool args
- `state.contact`
- `state.captain_v2_trace_current_input` (message text parsing)
- `state.captain_v2_trace_input` history (supports confirmation turns like `Yes`)
### E. `shopify_search_products` tool
### `shopify_search_products`
Params:
1. `query` (required string).
1. `query` (required)
Behavior:
1. Call `ProductsService#search_products`.
2. Format compact plain text list (max 10):
- title
- price
- availability
- product URL.
3. Deterministic fallback messages for each domain error.
1. Calls `ProductsService#search_products(query:, limit: 10)`.
2. Returns compact text list with title, price, availability, and product URL.
3. Returns safe fallback messages on domain/provider errors.
### F. `shopify_get_orders` tool
### `shopify_get_orders`
Params:
1. none (phase 1).
1. `email` (optional)
2. `phone_number` (optional)
Behavior:
1. Resolve contact email/phone from conversation state.
2. Call `OrdersService#orders_for_contact`.
3. Format compact plain text list (max 10):
- order name/date/total/status/admin URL
- top line items.
4. Deterministic fallback messages for each domain error.
1. Resolves identity through base-tool identity resolution.
2. Calls `OrdersService#orders_for_contact(email:, phone_number:, limit: 10)`.
3. Returns compact text list with order metadata, admin URL, and top line items.
4. Returns deterministic fallback messages for missing identity/no results/scope/provider errors.
## Gating Rules (Decision-Complete)
1. Tool visibility in UI:
- Hidden unless `captain_integration_v2` and Shopify connected.
2. Tool execution:
- If invoked when ineligible, return non-execution message (no crash).
3. Scenario validity:
- Existing `available_tool_ids`-based validation remains source of truth.
## Gating Rules
1. Shopify tools are exposed only when Captain V2 is enabled and Shopify is connected.
2. If tool execution is attempted while ineligible, tools return safe non-crashing messages.
3. Existing `available_tool_ids` validation remains source of truth for scenario/tool validity.
## Test Cases
## Current Constraints
1. Order lookup is email/phone-based only in this version.
2. No order-ID lookup path is enabled in this implementation.
3. Product and order responses are plain text lists for Captain tool output.
## Verification Coverage
### Service specs
1. Product search success with normalized fields.
2. Product no results.
3. Product missing scope.
4. Orders success by email.
5. Orders success by phone.
6. Orders missing identifier.
7. Orders no matching customer.
8. Provider error mapping for both services.
1. Product search success/no-results/scope/provider-error.
2. Product title-empty fallback keyword flow.
3. Orders success by email and by phone.
4. Orders missing identifier/no customer/provider-error.
### Tool specs
1. `active?` false when V2 disabled.
2. `active?` false when Shopify disconnected.
3. Product tool success formatting.
4. Product tool missing scope message.
5. Orders tool success formatting.
6. Orders tool missing identity message.
7. Orders tool no-result message.
8. Provider error safe messaging.
1. Active-state gating checks (`V2 off`, `Shopify disconnected`, eligible case).
2. Product tool success/error messaging.
3. Orders tool success/error messaging.
4. Orders identity extraction from:
- state contact
- current input text
- trace history
- explicit tool arguments overriding inferred values
### Model/gating specs
1. `available_agent_tools` hides Shopify tools unless eligible.
2. `agent_tools` includes Shopify tools only when eligible.
### Regression
1. Existing Shopify integration request specs still pass.
2. Existing Captain tools unaffected.
## Acceptance Criteria
1. With V2 enabled + Shopify connected + proper scopes:
- “Do you have sneakers?” triggers `shopify_search_products` and returns relevant products with links.
- “Where is my order?” triggers `shopify_get_orders` and returns recent order details.
2. With older Shopify install missing product scope:
- Product tool returns reconnect guidance; no crash.
3. With V2 off or no Shopify connection:
- Shopify tools are not shown and not executed.
## Assumptions
1. Enterprise-only feature is acceptable.
2. Captain V2 only.
3. Phase 1 excludes abandoned checkouts/cart.
4. Plain text output only.
5. English copy updates only where needed.
## `shopify_advanced.md` Content
Use this entire document as the initial content of `shopify_advanced.md` at repo root.
## Notes for Next Version
1. Optional order lookup by explicit order ID (feature-flagged).
2. Optional verified identity persistence to contact profile.
3. REST-to-GraphQL Shopify migration to address deprecation trajectory.
4. More end-to-end conversation-level regression coverage.