下面是一份可直接用于公司内部分享的英文 Knowledge Share。核心设计是：
每一级 AI 项目都继承前一级的 Evals，同时增加与新复杂度对应的评估层。
项目越复杂，不是替换旧指标，而是在共同的 Eval Core 上增加新的 Eval Packs。
Suggested duration: 25–30 minutes
Target audience: Product managers, AI engineers, data scientists, platform engineers, domain experts, security teams, and business owners
Not all AI projects have the same level of complexity.
A document extraction workflow, an internal chatbot, a tool-using agent, and a multi-agent system may all use large language models, but they fail in very different ways.
For example:
Therefore, a single generic metric such as “answer quality” is not sufficient.
The evaluation strategy must evolve with the architecture of the AI system.
However, this does not mean that every team should build a completely separate evaluation framework. Most foundational capabilities should be shared across the company.
The recommended model is:
Shared Enterprise Eval Core
+
Level-Specific Eval Packs
We can organize enterprise AI projects into four levels.
Level 1
AI-Enabled Workflow
Extraction, classification, summarization
↓
Level 2
RAG Application or Chatbot
Knowledge retrieval and conversational responses
↓
Level 3
Single Agent with Tools
Planning, tool selection and external actions
↓
Level 4
Multi-Agent or Agentic System
Multiple agents, delegation and complex coordination
Each level introduces additional uncertainty, autonomy and business risk.
| Level | Typical Architecture | Main Source of Complexity |
|---|---|---|
| Level 1 | Input → LLM → Structured output | Output correctness |
| Level 2 | User → Retrieval → LLM → Answer | Knowledge grounding |
| Level 3 | User → Agent → Tools → Actions | Decision and execution trajectory |
| Level 4 | Multiple agents → Tools → Shared state | Coordination and emergent behavior |
When an AI project moves from Level 1 to Level 2, we should not stop evaluating extraction accuracy, format correctness or latency.
Instead, we retain those evaluations and add retrieval and grounding evaluations.
The same principle applies at every level.
Level 1:
Output Evals
Level 2:
Output Evals
+ Retrieval Evals
+ Grounding Evals
+ Conversation Evals
Level 3:
All Level 1 and Level 2 Evals
+ Tool-Use Evals
+ Trajectory Evals
+ State-Change Evals
Level 4:
All Previous Evals
+ Coordination Evals
+ Delegation Evals
+ Multi-Agent Simulation
The higher the project level, the broader the evaluation surface becomes.
Every AI project should use a common enterprise evaluation foundation.
Each project must clearly define:
Example:
Given a warranty claim, vehicle information, repair notes and warranty policies, the system should determine coverage eligibility, provide supporting evidence and escalate ambiguous or high-risk cases for human review.
The company should maintain a common failure taxonomy.
Typical categories include:
Incorrect answer
Missing information
Unsupported claim
Wrong citation
Invalid format
Policy violation
Privacy exposure
Unauthorized action
Incorrect escalation
Tool failure
Excessive latency
Excessive cost
Individual projects may add domain-specific categories, but the high-level taxonomy should remain consistent.
This makes it possible to compare reliability across projects.
Every Eval case should follow a common schema.
{
"case_id": "WC-00125",
"project": "warranty-assistant",
"input": {},
"expected_behavior": {},
"reference_answer": null,
"slice_tags": [
"high_value_claim",
"policy_exception"
],
"risk_level": "critical",
"source": "production_anonymized",
"rubric_version": "v1.2"
}
Each project should maintain several datasets:
| Dataset | Purpose |
|---|---|
| Golden Dataset | High-quality cases labeled by domain experts |
| Regression Dataset | Previously discovered failures |
| Edge-Case Dataset | Unusual but valid scenarios |
| Adversarial Dataset | Security, policy and prompt-injection tests |
| Production Sample | Recent real-world traffic |
| Synthetic Dataset | Coverage expansion when real data is limited |
Synthetic data is useful, but it should not replace real production cases.
All projects should use a combination of evaluators.
Use code whenever the expected result can be objectively checked.
Examples:
Use domain experts for:
Use LLM Judges for scalable semantic evaluation.
Examples:
LLM Judges must be calibrated against human labels before being trusted.
Not every failure should have the same weight.
| Severity | Meaning | Example |
|---|---|---|
| Critical | Legal, safety or major financial impact | Unauthorized payment approval |
| High | Serious business or customer impact | Incorrect policy interpretation |
| Medium | Human correction required | Missing claim detail |
| Low | User-experience issue | Response is unnecessarily long |
A high average score must never hide critical failures.
For example:
Overall accuracy: 97%
Critical unauthorized actions: 3
This system should not be released.
Every experiment should capture:
Dataset version
Prompt version
Model version
Retrieval configuration
Tool version
Evaluator version
Rubric version
Temperature and model parameters
Latency
Token usage
Cost
Evaluation results
Without versioning, teams cannot explain why a model improved or regressed.
Every AI project should run automated Evals before deployment.
A release decision should include:
Example:
Critical failures = 0
Schema validity = 100%
Task success rate ≥ 95%
No critical slice regresses by more than 2%
p95 latency ≤ 8 seconds
Average cost within budget
Offline evaluation alone is not enough.
The production loop should be:
Production Trace
↓
Automated Online Evaluation
↓
Human Review for Selected Cases
↓
Failure Classification
↓
New Regression Test
↓
System Improvement
↓
CI/CD Validation
Every important production failure should eventually become a regression test.
Level 1 projects perform a bounded transformation from input to output.
Typical examples include:
The model normally does not retrieve external knowledge, call tools or make autonomous decisions.
Input
↓
Prompt and Model
↓
Structured or Text Output
Did the system generate the correct output in the required format?
For extraction and classification:
For summarization or generation:
Input:
Customer repair notes and claim documents
Expected output:
{
"vehicle_id": "12345",
"repair_date": "2026-06-15",
"claim_amount": 2850,
"repair_category": "engine",
"missing_information": []
}
Recommended Evals:
JSON validity = 100%
Required-field completion ≥ 99%
Critical field accuracy ≥ 98%
Overall field accuracy ≥ 95%
No sensitive data leakage
Level 2 projects retrieve enterprise knowledge and generate answers based on that knowledge.
Examples include:
User Question
↓
Query Processing
↓
Document Retrieval
↓
Context Assembly
↓
LLM Answer
Did the system retrieve the right information?
Is the final answer supported by the retrieved information?
Level 2 systems still require:
However, additional RAG-specific evaluations are required.
Recommended metrics include:
The goal is not only to determine whether the answer is wrong, but also whether retrieval caused the error.
Evaluate whether:
For multi-turn systems, evaluate:
A good RAG system should not answer every question.
Evaluate whether it:
User question:
Is an engine repair covered if the vehicle is outside the standard warranty period but has an extended coverage plan?
The system must:
Recommended Evals:
Correct policy retrieval
Relevant chunk Recall@K
Coverage answer correctness
Citation correctness
Groundedness
Completeness
Correct abstention
Multi-turn consistency
Relevant-document Recall@5 ≥ 95%
Citation accuracy ≥ 98%
Unsupported claim rate ≤ 2%
Correct escalation ≥ 95%
Critical policy errors = 0
A Level 3 project allows an AI agent to decide what actions to take and interact with external systems.
Examples include:
User Goal
↓
Agent Planning
↓
Tool Selection
↓
Tool Arguments
↓
Tool Execution
↓
State Update
↓
Continue, Retry, Stop or Escalate
Did the agent complete the task correctly, safely and efficiently?
The final answer alone is not enough.
Two agents may produce the same final message, but one may have called the wrong database, exposed sensitive information or modified the wrong record.
Level 3 still requires:
It also requires evaluation of the agent trajectory.
Evaluate whether the agent:
Evaluate:
Evaluate the complete sequence of actions:
Plan
Tool call
Observation
Updated plan
Next action
Final result
Important metrics include:
When an agent changes an external system, evaluate the resulting state.
For example:
The best evaluation is often not:
Did the agent say it completed the task?
It is:
Is the final system state correct?
Evaluate whether the agent:
The agent may:
Recommended Evals:
Claim decision correctness
Correct policy retrieval
Correct tool selection
Tool argument accuracy
Calculation accuracy
External state correctness
Approval-policy compliance
Recovery from API failures
Correct escalation
Step count
Latency and cost
Unauthorized actions = 0
Critical state-change errors = 0
Tool argument accuracy ≥ 99%
End-to-end task completion ≥ 95%
Correct escalation ≥ 98%
Duplicate action rate ≤ 1%
Level 4 projects contain multiple specialized agents that collaborate, delegate tasks and share state.
Examples include:
Coordinator Agent
↓
Specialist Agents
↓
Tools and Data Sources
↓
Shared Memory or State
↓
Review and Decision Agent
↓
Final Business Outcome
Did the entire system achieve the global objective through effective, safe and reliable coordination?
Every individual agent still requires:
Level 4 adds system-level coordination evaluations.
Evaluate whether:
Evaluate whether:
Evaluate:
Evaluate whether agents:
Test situations such as:
Multi-agent systems often require simulated environments.
A scenario may include:
Initial business state
User goal
Available agents
Available tools
Permission model
Injected failures
Expected final state
Maximum cost and steps
Evaluation should focus on the final environment state and the full event history.
Possible agents:
Recommended Evals:
Correct task delegation
Handoff completeness
Evidence consistency
Conflict resolution
Global decision correctness
Shared-state integrity
Duplicate-work rate
Agent-loop detection
System recovery
Total cost
Total processing time
Final claim-system state
Critical coordination failures = 0
Conflicting final actions = 0
Shared-state corruption = 0
Global task success ≥ 95%
Successful recovery from agent failure ≥ 90%
No uncontrolled loops
Execution remains within cost and step budgets
| Eval Area | Level 1 Workflow | Level 2 RAG/Chatbot | Level 3 Agent | Level 4 Multi-Agent |
|---|---|---|---|---|
| Output correctness | Required | Required | Required | Required |
| Schema validation | Required | Required | Required | Required |
| Business-rule compliance | Required | Required | Required | Required |
| Cost and latency | Required | Required | Required | Required |
| Retrieval quality | Optional | Required | Required | Required |
| Groundedness | Optional | Required | Required | Required |
| Citation accuracy | Optional | Required | Required | Required |
| Multi-turn consistency | Optional | Required | Required | Required |
| Tool selection | Not applicable | Limited | Required | Required |
| Tool argument accuracy | Not applicable | Limited | Required | Required |
| Trajectory evaluation | Not applicable | Limited | Required | Required |
| External state validation | Not applicable | Not typical | Required | Required |
| Delegation quality | Not applicable | Not applicable | Not typical | Required |
| Handoff quality | Not applicable | Not applicable | Not typical | Required |
| Shared-state consistency | Not applicable | Not applicable | Limited | Required |
| Coordination and conflict resolution | Not applicable | Not applicable | Limited | Required |
| Simulation and chaos testing | Limited | Limited | Recommended | Required |
The key message is:
Higher-level systems require broader evaluation, but they do not eliminate the need for lower-level tests.
The company should build one shared Eval Core and several reusable Eval Packs.
Enterprise Eval Platform
┌──────────────────────────────────────────────────────────────┐
│ Shared Eval Core │
│ │
│ Dataset Registry │
│ Human Annotation and Review │
│ Evaluator SDK │
│ LLM Judge Management │
│ Experiment Tracking │
│ Trace and Log Storage │
│ CI/CD Release Gates │
│ Production Monitoring │
│ Dashboards and Governance │
└──────────────────────────────────────────────────────────────┘
│ │ │
▼ ▼ ▼
Workflow Pack RAG Pack Agent Pack
│
▼
Multi-Agent Pack
The shared platform should provide:
Dataset management
Evaluator execution
Prompt and model versioning
Experiment comparison
Human review workflow
Judge calibration
Tracing
Release gates
Online monitoring
Failure analytics
Access control
Audit history
Reusable evaluators for:
Reusable evaluators for:
Reusable evaluators for:
Reusable evaluators for:
Regardless of project level, every AI project should follow the same lifecycle.
Do not begin with model metrics.
Begin with:
Capture:
Input
Output
Prompt version
Model version
Retrieved documents
Tool calls
Tool results
Agent steps
Final state
Latency
Tokens
Cost
User feedback
The exact trace becomes richer at higher project levels.
Start with approximately 50–200 high-quality real examples.
Include:
For each result, ask:
Pass or fail?
Why?
What type of failure occurred?
What should the correct behavior be?
How severe is the failure?
Do not use an LLM Judge for something that code can verify reliably.
Compare Judge results against human labels.
Measure:
For every meaningful change:
Current Production Version
vs.
Candidate Version
Compare quality, safety, cost and latency.
Block releases when:
Use:
Every serious production failure should become a permanent test case.
The same business domain can evolve through all four levels.
The system extracts:
Claim amount
Repair date
Vehicle ID
Repair category
Customer information
Primary Evals:
Field accuracy
Schema validity
Missing-field detection
Classification accuracy
The system retrieves warranty documents and answers coverage questions.
Additional Evals:
Policy retrieval
Groundedness
Citation accuracy
Correct abstention
Conversation consistency
The agent queries systems, calculates coverage and updates claim status.
Additional Evals:
Tool selection
Tool argument accuracy
Calculation accuracy
Workflow completion
External state validation
Authorization compliance
Specialized agents investigate policies, vehicle history, fraud risk and financial impact.
Additional Evals:
Delegation quality
Handoff completeness
Evidence consistency
Conflict resolution
Shared-state integrity
Global task success
This example demonstrates that the Eval Framework grows with the project.
It does not restart at every level.
| Role | Primary Responsibility |
|---|---|
| Product Manager | Define user success and product acceptance criteria |
| Business Owner | Define business value and risk tolerance |
| Domain Expert | Define rubrics and label difficult cases |
| AI Engineer | Build prompts, agents and evaluators |
| Data Scientist | Analyze metrics, slices and evaluator quality |
| Data Engineer | Build dataset and feedback pipelines |
| Platform Engineer | Build tracing, CI/CD and shared Eval infrastructure |
| Security and Legal | Define privacy, safety and compliance requirements |
| Human Operations Team | Review escalations and production failures |
Evals should not be owned only by the AI engineering team.
They are a cross-functional product responsibility.
Run:
Unit Evals
Regression Evals
Safety Evals
Cost and latency checks
Review:
New production failures
User negative feedback
Judge disagreements
Critical slices
Update:
Golden Dataset
Production Dataset
Failure Taxonomy
Thresholds
Evaluator prompts
Perform:
Red-team testing
Policy review
Risk review
Judge recalibration
Architecture-level failure testing
Run:
Full baseline comparison
Repeated-run consistency tests
Slice-level regression analysis
Canary deployment
Production monitoring
Deliverables:
Project-level definitions
Shared task contract template
Risk and severity framework
Failure taxonomy
Dataset schema
Tracing standard
First golden dataset
Human evaluation rubric
Start with one Level 1 or Level 2 project.
Deliverables:
Deterministic evaluator library
Calibrated LLM Judges
Experiment runner
Version tracking
Regression suite
CI/CD release gates
Eval dashboard
Deliverables:
Online evaluation sampling
Human review queue
Production feedback pipeline
Automatic regression-case creation
Risk dashboards
Governance meetings
RAG and Agent Eval Packs
After the shared foundation is stable, expand to Level 3 and Level 4 projects.
An overall score hides the cause and severity of failures.
Use multiple dimensions and slices instead.
The first question should not be:
Which Eval platform should we buy?
The first question should be:
What failure would create the greatest business harm?
For RAG systems, evaluate retrieval.
For agents, evaluate tools and state changes.
For multi-agent systems, evaluate coordination.
An LLM Judge is another model and can also be wrong.
It must be evaluated against domain-expert labels.
A system may perform well overall but fail on:
High-value claims
Policy exceptions
New product types
Non-English input
Long documents
Security-sensitive requests
Always evaluate important slices separately.
Evals should operate continuously across:
Development
Experimentation
CI/CD
Deployment
Production monitoring
Incident review
Different levels of AI projects require different evaluation strategies.
Higher-level systems inherit lower-level Evals and add new evaluation layers.
The company should build one shared Eval Core instead of separate frameworks for every project.
Level-specific Eval Packs should cover:
Level 1: Output correctness
Level 2: Retrieval and grounding
Level 3: Tools, trajectories and state changes
Level 4: Coordination, delegation and system behavior
Real production failures, domain-expert judgments and continuous feedback are more valuable than generic benchmark scores.
The final goal is not to prove that an LLM is intelligent.
The final goal is to demonstrate that the entire AI system is:
Useful
Correct
Reliable
Safe
Cost-effective
Observable
Governable
A successful enterprise Eval Framework should answer three questions:
The most effective architecture is:
One Shared Enterprise Eval Core
+
Level-Specific Evaluation Packs
+
Continuous Production Feedback
This approach allows the company to support simple AI workflows today while building a scalable foundation for future agentic systems.
这份内容已经按照约 15 页 Knowledge Share 的逻辑组织，可以直接用于讲解，也能进一步压缩成 PPT。最值得在演示中重点可视化的是“四级项目模型”“Shared Eval Core + Eval Packs”和 Warranty Claim 从 Level 1 演进到 Level 4 的案例。