Phil Schmid 在 aiDotEngineer 博览会上指出 vibe-checking agent 技能在生产中会失败，并介绍了构建可靠自动化评估的方法：添加负面测试、限制文件大小、使用正则断言和消融测试
AI 摘要
Phil Schmid 在 aiDotEngineer 博览会上指出 vibe-checking agent 技能在生产中会失败，并介绍了构建可靠自动化评估的方法：添加负面测试、限制文件大小、使用正则断言和消融测试。
推荐理由常规快讯，保留列表
原文
At the @aiDotEngineer World's Fair, I gave a talk on why vibe-checking agent skills breaks in production and how to build reliable automated evals before your users find the bugs. I talked about:<br><br> 🎯 Why vague descriptions cause skill failures and how adding negative test cases prevents keyword hijacking on unrelated prompts <br>✂️ Why skill files over 500 lines degrade model reasoning<br>⚡ How to validate outcomes using simple, millisecond regex assertions across 10 to 20 production prompts. <br>🗑️ How to run ablation tests (with vs. without skills) to know when models have caught up and you can retire skills.<br><br>Check out the full talk and resources below:<br><video width="1920" height="1080" src="https://video.twimg.com/amplify_video/2079204259212509184/vid/avc1/1920x1080/mfOv_SY-88X5GJby.mp4?tag=29" controls="controls" poster="https://pbs.twimg.com/amplify_video_thumb/2079204259212509184/img/DUu50gwn1AVcfPtv.jpg"></video>
讨论
暂无评论。