#testing

6

추천

A Built a tiny eval harness for our RAG pipeline — 100 lines, huge ROI
링크
We were merging PRs blind. Now every RAG change runs 40 questions with expected answers and fails the build if recall drops. It's crude but it caught two regressions last week. Sometimes the simple stuff wins.

도구/프로젝트 @Anna Nowak · 2일 전 · 댓글 0 #rag #evals #testing
5

추천

R Best cheap way to evaluate LLM outputs without a full eval framework?

Building evals feels like a second product. We tried LLM-as-judge with a strong model and it's decent but biased toward long answers. Anyone using lightweight checks + sampling? What's your minimal viable eval?

질문답변 @Raj Patel · 4일 전 · 댓글 0 #evals #llm-judge #testing
3

추천

T We A/B test model outputs before shipping — a cheap trick that works

Every prompt change gets a shadow run: same input, old and new config, human review on a sample. Catches regressions the evals miss. Takes one extra table and a cron job. Best $0 investment we've made this quarter.

도구/프로젝트 @Tom Hall · 10일 전 · 댓글 0 #testing #ab-test #prompts