Every accelerator, lender and consultancy eventually builds a founder assessment, and almost all of them fail in the same place. The tool starts as a genuine attempt to see clearly: a set of questions, a score, a band, some honest language about what the score means. Then someone notices that people who complete it are unusually warm leads, and the tool acquires a second job. The questions stay the same, the scoring stays the same, and something subtler changes: the result now has to lead somewhere commercial. Within two revisions the low band has stopped saying “you are not ready for this” and started saying “you need this most of all,” which is the precise moment the diagnostic stops being a diagnostic and becomes a sales instrument wearing a lab coat.
Key Takeaways
- A diagnostic that awards the prize for completing it is no longer a diagnostic. Completion is an act anyone can perform.
- The integrity test is what the lowest band says. If the tool cannot tell someone “not yet,” it cannot tell anyone anything.
- Self-scored assessments have a known bias direction, and the founders who score themselves generously learn least from the exercise.
- Scoring beats filtering, because a filter loses the strong candidate with the weak line item silently while a rubric surfaces them by name.
- A score you cannot explain to the person it declined is not a score, it is a decision with decoration.
- Weight the criteria against observed outcomes, not intuition. Completion data beats interview impressions every time.
What is the test for whether a diagnostic is honest?
Read the lowest band and ask what it recommends. If the tool tells a founder scoring in the bottom quartile that the paid programme is exactly what they need, it has no diagnostic content, because it produces the same recommendation regardless of input. A tool whose output does not vary with its input is not measuring anything.
The honest version says something harder: that at this stage, a course will not move these numbers, that only proof will, and that the founder should go and take one payment from one customer and come back. That advice loses revenue. It is also the only version that makes the top band’s recommendation credible, because a tool that can say no is a tool whose yes carries information.
The same logic governs the moment of allocation. If completing an assessment automatically awards a scarce, paid place, then the assessment has been converted into a queue and the scarce resource is being allocated by typing speed. The design correction is to separate the act from the outcome: completing puts a founder into the review, and a human decides afterwards against published weights.
How much should you trust a self-scored answer?
Less than the number suggests, and in a predictable direction. Founders with strong, aligned teams reliably over-score their business model, because a capable team makes a weak model feel workable far longer than it should. Founders with polished language score better than founders with better economics, because articulacy is easier to perform than margin. And confident round numbers are systematically less reliable than hedged specific ones.
That last observation is worth building into the marking scheme rather than merely noting. In our own capability corpus the answers that read “I do not know yet, here is what I would measure first” came from founders who had thought harder than those who supplied a clean, unsourced profitability figure. A rubric that rewards confidence over calibration will select for the wrong trait, and it will do so invisibly, because confident answers are more pleasant to read.
The mitigations are unglamorous. Ask for numbers with units and a period attached. Ask what the founder would measure first if they do not know. And treat any pillar scored near perfect as a prompt for a question rather than a result, particularly when it is the pillar carrying the least weight.
Why score rather than filter?
Because filters fail silently and rubrics fail loudly. A filter on ability to pay removes, at step one, the registered and trading venture whose founder wrote a careful request for a scholarship, and nobody ever sees the name. A weighted rubric scores her venture before it reads her wallet, places her fifth, and forces an explicit decision about whether the scholarship budget exists. The founder is the same in both systems. Only one of them can find her.
This is also what makes a programme defensible. A rejected applicant who asks why can be told: these were the criteria, this is what you scored, here is the dimension that was weak. That conversation is survivable. The alternative, that places went to whoever submitted first, is not, and it should not be, because it is indefensible on its face.
The final rule follows from the same place. Weights should come from observed outcomes rather than conviction, which means holding the historical file and checking what actually predicted completion. Ours said business status at entry predicted it three-fold, and that finding overruled several confident opinions, including some of mine.
FAQ
Is it wrong to generate leads from an assessment?
No. It is wrong to let the lead-generation purpose change the scoring or the recommendation. Keep the diagnosis honest and the follow-up can be commercial.
What if the low band never converts?
It should not, at that stage. Route it to a lighter product matched to the stage and readmit later. A programme with only one product will keep mis-selling it.
Should scores be shown to applicants?
Show the score and its components. Anything you would not say out loud to the person it excluded should not be in the model.
How often should weights be revisited?
Annually, against the outcomes of the intervening cohorts. Treat the current weights as the best available evidence, not as settled.
Related Reading
- Who actually finishes: what 99 alumni say about selection
- What a founder credit score should actually measure
- Founders do not lack data, they lack a decision the data could change
- Dignity has a price architecture
Sources and Evidence
- Blacksmith, N. and McCusker, M. E., Data-Driven Decision Making in Entrepreneurship, on assessment validity and the limits of self-report.
- Internal capability-assessment corpus, 2026, in which hedged numerate answers proved more diagnostic than confident round figures.
- Internal alumni outcomes dataset, 99 participants 2022 to 2026, used to weight selection criteria against observed completion rather than intuition.