Who wins, the person posting picks or the model outputting probabilities? Handicapper vs model is not a fair fight until both are measured the same way. You will learn to build a shared set of events with the same cutoff times and the exact odds visible then, and to report coverage and abstentions instead of cherry-picking best plays. A worked example shows why a 60 percent pick record says little about calibration, and how a flat 60 percent forecast on ten events earns a Brier score of 0.24. Price matters too: paying -200 needs a 66.67 percent hit rate just to break even. Data leakage and selective publishing round out the checks.
Learning objectives
- Develop a reproducible evaluation procedure for the technique.
- Work through the concrete examples and identify limitations.
- Complete the exercises and interpret findings without overstating certainty.
