QA Agent

The QA agent validates the output of the Processing agent, checking coverage thresholds and chunk quality.

Tools

Output: QADecision

class QADecision(BaseModel):
    verified: bool
    structuring_coverage: float
    chunking_coverage: float
    end_to_end_coverage: float
    empty_chunk_count: int
    chunks_missing_topics: int
    recommendations: list[QARecommendation]
    reasoning: str
 
class QARecommendation(BaseModel):
    issue: str
    action: str
    parameter: str
    suggested_value: str

How It Works

1. Calls check_coverage to compute word-level coverage between the source text and chunks
2. Calls check_chunks to inspect chunk quality:
   • Empty chunks (less than 10 characters)
   • Chunks missing topic labels
   • Chunks missing key_concepts
3. If quality is acceptable, sets verified=True
4. If issues found, returns QARecommendation objects suggesting parameter adjustments

Example Recommendation

{
  "issue": "3 empty chunks detected",
  "action": "set",
  "parameter": "min_tokens",
  "suggested_value": "50"
}

Or for larger quality issues:

{
  "issue": "Low structuring coverage (0.82)",
  "action": "change",
  "parameter": "preset",
  "suggested_value": "legal"
}

Note: check_chunks does not call any distillcore API — it performs pure Python inspection of the chunk data.