CBLE Score Estimator · October 2026
Methodology & accuracy
How public survey answers become a score estimate—and what historical results suggest about its accuracy.
What the estimator measures
The CBLE Score Estimator compares an entered answer sheet with a provisional key built from the public October survey. It counts matches across all 80 questions, then uses April 2026 score errors to estimate a range and a conditional chance of passing.
The provisional key is based on weighted agreement among respondents. It is not an official CBP key, and a question’s vote-support percentage measures agreement rather than the probability of being correct.
How the weights are set
The model uses the top 10% of respondents by agreement, with adjustments for consistency and similar answer patterns. The selection and weights are recalculated for each question.
-
Start with sufficiently complete responses
The source data must reproduce the published survey counts. The weighted pool then retains respondents with at least 72 valid answers out of 80.
-
Rank agreement on the rest of the exam
Each respondent is compared with the majority among everyone else. Their own vote is removed from that comparison, and the question being estimated is excluded. An answer therefore cannot directly increase its own weight.
-
Select the top 10% and moderate their weights
Agreement rates are smoothed and converted into weights, then moved halfway toward equal voting. The top 10% receive voting places; ties at the cutoff share the remaining places. Respondents below the cutoff contribute no votes to that question.
-
Discount similar answer patterns
Respondents matching on at least 80% of their other jointly answered questions are treated as similar peers. Each vote is divided by 1 + 0.1 × similar-peer count. Five similar peers, for example, reduce a vote to two-thirds of its original weight.
-
Build the provisional key
Adjusted votes are added for A, B, C, and D. The leading answer becomes the provisional choice. A match to tied leaders receives fractional credit, and the total score is rounded to a whole number.
These settings were developed through comparisons on April responses and official grades. October uses the same rule, with rankings and weights calculated from October’s own response pool.
Weight formula and data filters
For respondent i, let Hᵢ be the sum of majority-agreement credits and Nᵢ the number of valid answers, both excluding the question being estimated. The implementation uses:
rᵢ = ln(3pᵢ / (1 − pᵢ))
wᵢ = max(0.05, 0.5 + 0.5 × rᵢ / median(r))
final vote = cutoff share × wᵢ / (1 + 0.1 × similar-peer count)
Selection uses 10% of eligible respondents, rounded up, with a minimum of three places. Tied majority choices receive fractional agreement credit. The logarithmic transform measures agreement relative to a four-choice baseline. The median scaling puts a median respondent at weight 1; the halfway adjustment moderates the spread. Cutoff share is 1 above the selection threshold, 0 below it, and fractional for a tie at the threshold. A missing answer contributes no vote.
Before reconciling with the published counts, the source import excludes rows with fewer than 40 valid answers, at least 70 repetitions of one answer letter, or agreement with published leading choices on fewer than 35% of the 80 questions. The weighted pool applies the stricter 72-answer requirement. These filters favor sufficiently complete, consensus-aligned responses and can also exclude legitimate outlying answer patterns.
Expected accuracy
April’s calibration results suggest a useful reference of about one or two questions of error. Among the 231 sheets used to set the model’s error bands, 41.1% matched the official score exactly, 82.3% were within one question, and 97.0% were within two.
October accuracy has not been independently established. The method was developed on April, so these historical percentages are a reference—not a validated prediction of October performance.
Reading the score bands
The bands add historical error intervals to the estimated score. For example, an estimate of 62 produces an 80% band of 61–63 and a 95% band of 61–64.
| Band | Offset | Calibration coverage | Internal-check coverage |
|---|---|---|---|
| 80% | −1 to +1 | 190 / 23182.3% | 51 / 5887.9% |
| 95% | −1 to +2 | 224 / 23197.0% | 56 / 5896.6% |
The 58-sheet check uses different answer sheets from the same April exam. It tests the bands internally; it does not establish accuracy on a new exam.
Calibration method and observed errors
The analysis uses 289 unique, complete April sheets: 231 to construct the error distribution and 58 for the internal check. Each evaluated pattern and its exact duplicates were excluded from its source pool. All 80 questions and official special-grading decisions are included.
Error means official grade minus estimated key matches. Positive errors are underestimates; negative errors are overestimates.
| Official grade relative to estimate | Sheets | Share of 231 |
|---|---|---|
| 1 point lower (−1) | 50 | 21.6% |
| Exact match (0) | 95 | 41.1% |
| 1 point higher (+1) | 45 | 19.5% |
| 2 points higher (+2) | 34 | 14.7% |
| 3 points higher (+3) | 7 | 3.0% |
Each band is the shortest observed error interval covering at least its stated fraction of calibration sheets. Bands are clipped to 0–80. The plotted bell curve approximates the distribution visually; the reported bands use the discrete errors above.
Comparing the two estimates
The results page compares the weighted key with an ordinary majority key, where each survey response receives one equal vote. Their difference shows sensitivity to weighting; it is not a confidence interval. Tied leading choices split match credit equally.
A question is labeled divided when its leading choice receives less than 75% of weighted support. The answer grid shows probabilities fitted to April’s official answers, with green at 75% or above, gray from 25% to below 75%, and tan below 25%. These cutoffs are presentation rules. The probability model below is separate from the historical total-score bands. Question details also show the underlying vote support for each choice.
The historical score bands and passing calculation use April’s total-score errors. They do not widen automatically when October contains more divided questions.
Fitting the probability of each answer
Historical comparison added: four archived exams plus April 2026 now provide 400 question outcomes. Whole-exam testing evaluates 395 ordinary questions; five special-grading outcomes remain separate. The historical model uses equal votes because the archive does not preserve verified respondent alignment. It is available as a comparison in each answer’s details, while the main bars retain the weighted April fit described below. Explore the tests, mistakes, and model comparisons.
Experimental model fitted to April—not a preset error budget. The same top-10% weighting rule produces vote shares from 318 eligible April responses. The official key supplies 78 ordinary single-answer outcomes. Q19 remains in training as an observed consensus mistake. Q61 (B and D accepted) and Q73 (universal credit) are excluded from this single-answer probability fit; they remain part of historical total-score grading.
For each question, the model converts its four weighted vote shares into four probabilities. It learns how strongly vote margins should separate the probabilities and includes a uniform error component so even unanimous agreement can be wrong. The published prediction averages across plausible parameter values rather than using only one best fit.
β controls sensitivity to vote margins; ε allows for errors not explained by those margins. Their posterior distribution is fitted against April’s official answers. The posterior mean of ε is about 3.0%; this is a uniform-mixture parameter, not a claim that every question has a 3.0% chance of consensus failure. The modeled chance the leader is wrong is one minus its displayed probability.
What the internal test shows
Leave-one-question-out testing withholds a question’s official answer and fits on the other 77. All four candidate choices stay together. The hundreds of survey responses are inputs to each question’s votes; they are not hundreds of independent official-answer outcomes.
| Model | Log loss | Multiclass Brier score |
|---|---|---|
| Fitted margins and error allowance | 0.112 | 0.031 |
| Raw weighted vote shares | 0.274 | 0.127 |
| Same historical probability for every leader | 0.095 | 0.026 |
| Uniform 25% per choice | 1.386 | 0.750 |
The fitted model improves on raw vote shares, but the simpler leader-only baseline performs better on both measures. That baseline estimates the leading choice’s success rate from the remaining questions with one prior success and failure, then spreads the remaining probability equally across the other choices. April therefore does not establish that the more detailed margin model is superior.
The important failure is retained: the weighted leader is correct on 77 of 78 ordinary April questions. With Q19 withheld, the model assigns 98.9% to C, but the official answer is A. The model accommodates consensus mistakes; it has not shown it can identify which strong consensus is wrong before seeing the key.
Regularization, sensitivity, and reproducibility
The default priors are half-normal(σ = 10) for β and Beta(1, 9) for ε. Numerical integration uses β from 0 to 60 in steps of 0.25 and ε from 0 to 1 in steps of 0.005, with trapezoidal endpoint weights. These priors are assumptions that prevent the almost-perfect April key from producing unlimited confidence. They are not an assumed number of wrong October answers.
Sensitivity checks use β prior scales of 5 and 20, plus a uniform Beta(1, 1) prior for ε. The range shown on each question spans those alternatives and the default. It is not a confidence interval. With the current October snapshot, C on Q7 varies from about 69% to 89%; that variation reflects how little one exam constrains the model.
The weighting rule itself was previously developed using April. Withholding questions from the probability fit does not undo that earlier model selection, establish that responses preceded the official key, or validate transfer to another exam. Special grading cannot be reliably predicted from the two April examples.
Anonymous April question evidence and withheld predictions and deployed model parameters are available for inspection. The repository’s calibration script can reproduce the fit from the aggregate evidence without publishing individual response sheets.
The passing estimate
The model adds each historical error to the estimated score and counts how often the result reaches at least 60 correct answers. One hypothetical pass and one hypothetical fail are added so a limited sample cannot produce 0% or 100%.
Passing calculation
Modeled passing chance = (historical passing cases + 1) / (231 + 2)
The calculation assumes April’s score errors transfer to October. Smoothing moderates numerical extremes; it cannot account for every unseen error or correct survey bias.
What can affect the result
- Shared mistakes. Strong agreement can still be wrong. Selecting the top 10% can reinforce a popular error or miss a correct minority answer.
- Survey participation. Voluntary submissions, missing answers, and similar response patterns can affect the key. A participant’s earlier survey submission may remain in the pool; the app’s aggregate data cannot remove it individually.
- Limited independent exams. The top-10% weighting settings were selected using April. The separate equal-vote probability model now has five-exam testing, which does not validate the weighted pipeline. It is unverified that all responses preceded publication of the official keys, which could make historical accuracy look stronger.
- Official grading. Multiple accepted answers or universal credit can change the final grade.
A stronger accuracy claim requires testing a fixed method on additional exams whose official answers were not used to choose its settings.
Sources & privacy
- April response sheet — historical answer patterns used in development and calibration.
- CBP April 2026 answer key — reference for official April grading.
- October public survey — source for the current provisional key.
- Published model evidence — aggregate votes, snapshot metadata, and the historical error counts used by the calculator.
- Five-exam evaluation — anonymous evidence, published key references, whole-exam tests, and retained consensus mistakes.
The calculator shows its evidence snapshot date and entry count. Only aggregate survey evidence is served. PDF parsing and scoring happen in the browser; entered PDFs and answers are not uploaded or saved.