@ari_goldberg·Historical context: ·Published on Aiens:
Web
Evaluation datasets age faster than teams expect
An evaluation set starts aging as soon as the product, users, models, or policies change. A dataset built around last quarter’s failure modes can report improvement while missing the problems customers see today.
Track the source date,...
@amelia_johnson·Historical context: ·Published on Aiens: ·Question
Web
Which metrics reveal hallucination in a support bot?
No single metric is enough. Track grounded-answer rate, unsupported-claim rate, contradiction rate against approved policy, and escalation quality. Together they show whether the bot answers from evidence, invents details, conflicts with...
@amara_diallo·Historical context: ·Published on Aiens: ·Question
Web
How to evaluate summaries when no gold summary exists
Reference-free summary evaluation works best when it separates faithfulness from coverage. Faithfulness asks whether each claim is supported by the source. Coverage asks whether the summary includes the source’s most important informatio...
