Verifiers - Fireside Chat with Will Brown #SFTechWeek
Hosted by
:◗
:◗91/100 spots left
No slides, no pitches, just war stories, patterns and anti patterns from the engineers who built RL verifiers.
In RLVR, the policy learns whatever the verifier rewards. If the verifier is leaky, exploitable, or miscalibrated, you’re training on lies. This fireside is a candid, practitioner-level dive with builder who have shipped multiple verifiers across tasks (safety, code, tool use).
We’ll hit:
- What a good verifier actually is: pairwise vs scalar scoring, sparse rewards, decomposition etc.
- Generalization under shift: preventing policy-chasing, leakage, and “reward hacking by spec ambiguity.”
- Calibration & coverage: thresholds, selective prediction, human-in-the-loop only where it moves expected utility.
- Sparse reward tasks: how to design verifiers when feedback is naturally scarce - detecting rare successes, using proxy signals without collapsing to spec drift, and tricks like trajectory-level verification or hierarchical reward shaping.
- If time permits: RL environments and how verifiers are the key.
Non-consensus takes we’ll debate:
- Verifiers should abstain a lot more (prevent the model from training on the wrong signal)
- Train/Configure verifiers on artifacts (traces/tests), not just text.
- Optimize for uncertainty quality, not mean score.
Who should come: folks shipping LLMs, agents, RLVR stacks; anyone who’s been burned by a “good” reward model.
Leave with a concrete checklist, sample specs, and a playbook to make your verifier the bottleneck on truth.
This event is a part of #SFTechWeek—a week of events hosted by VCs and startups to bring together the tech ecosystem. Learn more at www.tech-week.com.
Guest List
◕‿◕
˜▾˜
*⏖*