RAGAS

Headlines · 2

1. 2h
   A Practical Guide to AI Agent Evaluation Toolchains

   A Juejin tutorial organizes AI agent and application evaluation into four approaches: manual review, custom scripts, evaluation frameworks, and managed platforms. It compares tools including Promptfoo, DeepEval, LangSmith, RAGAS, and Langfuse, while advising teams to define datasets, metrics, and scorers before selecting execution tools. The article also warns about licensing, judge-model costs, vendor lock-in, and the limits of automating poorly defined evaluations.

   掘金 · 人工智能 · 🔥 5
2. Nov 1
   A Task-Level Framework for Evaluating AI Agents

   The ETF Agent evaluation framework argues that a green test only confirms that a check passed under defined conditions, not that the user’s task was completed. It recommends separating runtime behavior, evidence retrieval, content judgment and user outcomes, while preserving failed, inconclusive and corrected observations so model and system quality can be compared on a valid denominator. The article combines practical guidance on RAG, traces, model judges, sub-agent contributions and cost with historical examples from its own development process.

   掘金 · 人工智能 · 🔥 0

Experience and discussion from the community

Share my RAGAS experienceAsk about RAGAS