Troubleshooting and KPIs

Purpose

This guide provides a practical troubleshooting model and KPI framework for operating AVELIN products in production business environments.

Operating Goals

Restore critical business workflows quickly and safely
Minimize repeated incidents through evidence-based root-cause correction
Measure platform performance and governance quality with consistent KPIs
Improve decision quality for operations and leadership reviews

Incident Severity Classification

Severity	Business Impact	Initial Response Priority
SEV-1	Platform-wide outage or critical workflow stoppage	Immediate cross-functional response
SEV-2	Major degradation for high-value workflows	Rapid containment and owner escalation
SEV-3	Localized functionality degradation	Planned remediation with active monitoring
SEV-4	Minor issue with limited user impact	Backlog correction and trend tracking

Troubleshooting Workflow

Step 1: Confirm Business Impact and Severity

Identify affected user groups and workflows
Classify impact: localized, functional, or platform-wide
Assign incident severity based on business-criticality impact

Step 2: Identify Affected Product Area

AVELIN-GPT (interface and user interaction)
AVELIN-API (cascading and orchestration)
AVELIN-MCP (integration superpowers)
Y-RAY (search and analysis intelligence)
Trust and perimeter modules (access and policy controls)

Step 3: Validate Health and Signals

Check product readiness and continuity status
Inspect incident diagnostics and repeated error patterns
Confirm dependency and policy integrity for the failing path
Verify whether issue pattern is isolated or repeating across similar workflows

Step 4: Stabilize and Verify

Apply targeted remediation by product area
Avoid broad resets when precision recovery is possible
Validate end-to-end business workflow restoration
Confirm business owner acceptance before incident closure

Step 5: Capture Root Cause and Prevention

Document direct cause, contributing factors, and detection gaps
Define prevention actions with owners and deadlines
Feed learnings into runbooks, guardrails, and KPI reviews

Common Issue Matrix

Symptom	Likely Product Area	First Checks	Typical Remediation
User cannot access workspace	Perimeter or trust layer	Access policy, identity path, secure entry status	Restore access policy path and verify role controls
Slow or unstable responses	AVELIN-API	Cascading health, token pressure, Cross-Model MoE tier selection, tier pressure	Rebalance cascading policy and restore orchestration health
Actions not executing	AVELIN-MCP	Connector authorization and execution status	Revalidate integration permissions and action pathways
Research output lacks depth	Y-RAY	Retrieval scope and evidence path quality	Expand retrieval depth and rerun evidence-first synthesis
Workflow interruptions under load	Cross-product	Critical-path saturation and priority handling	Apply stabilization thresholds and protect key workflows
Unexpected policy blocks	Trust and governance layer	Access role path, policy exception state, workflow sensitivity class	Correct policy mapping or exception scope and retest controlled path
Cost spike without quality gain	AVELIN-API and governance	Cascading policy alignment, workload tagging, tier distribution, Cloud Token Factory efficiency	Rebalance cascading policies and tighten governance thresholds — verify optimal tier selection

Incident Closure Checklist

Service health restored for affected workflow paths
Business owner confirms workflow usability
Root-cause statement documented
Prevention actions assigned and tracked
Relevant dashboards updated with incident classification

KPI Framework

Reliability KPIs

Availability by product area
Mean time to detect (MTTD)
Mean time to recover (MTTR)
Incident recurrence rate
Critical-workflow restoration time

Productivity KPIs

Time-to-complete high-value workflows
Weekly active users by business function
Action completion rate from AI-assisted workflows
User confidence in response usefulness
Cross-team handoff success rate

Intelligence Quality KPIs

Citation-backed output ratio for Y-RAY workflows
Debate Mode usage for complex decisions
Rework rate for strategic outputs
Corporate Brain reuse rate across teams
Decision-owner confidence for high-impact workflows
Cross-Model MoE cascading quality — tier alignment
Cost efficiency vs premium alternatives — Intelligence up to 85%, Coding up to 86%, Agentic up to 87% savings

Governance and Risk KPIs

Trust policy violation trend
Access-governance compliance rate
Token governance efficiency trend
Audit readiness score
Policy exception aging and closure rate
Data-residency conformance for sensitive workflows

KPI Interpretation Guide

Reliability Signals

Rising MTTD indicates detection weakness and monitoring gaps
Rising MTTR indicates escalation or recovery execution issues
High recurrence indicates root-cause correction is incomplete

Productivity Signals

Falling completion time with stable quality indicates healthy adoption
Faster throughput with rising rework indicates quality controls need tuning
Low handoff success indicates cross-team workflow design problems

Governance Signals

Growing policy exceptions indicate control friction or policy mismatch
Poor audit readiness indicates evidence gaps in daily operations
High violation trend indicates need for trust-control reinforcement

Reporting Cadence

Cadence	Focus	Primary Audience
Weekly	Reliability and urgent risk indicators	Operations and service owners
Monthly	Productivity, intelligence quality, governance trends	Platform, security, and business leads
Quarterly	Strategic value realization and maturity progression	Executive leadership and governance boards

Dashboard Design Suggestions

Separate real-time incident dashboard from monthly trend dashboard
Show KPI trend direction, not only current values
Segment KPIs by business function for targeted corrective action
Keep one shared executive view for cross-product outcomes