What a Gender-Aware Selection Rubric Looks Like in Practice

Our own selection panels have yet to report a single stage of their results by gender. When twelve women-led businesses pitched at the eighth BK Urumuri Initiative in Kigali in November 2024, judges scored them on criteria including market analysis, team dynamics, investment potential and financial management. The newspaper account lists the criteria without weights or the evidence a founder had to bring. A gender-aware selection rubric supplies both. It fixes criteria and weights before the first application is read, scores what the business has done using records women hold, has reviewers score one criterion across the whole pile at a time, and reports results by gender at each stage.

Key Takeaways

• In a Yale experiment, evaluators rated education as more important for a police chief when the educated applicant was a man; committing to criteria first removed the gap.
• When Canada’s main health research funder reviewed the researcher instead of the research, women’s success gap widened from 0.9 to 4.0 percentage points across 23,918 applications.
• In Uganda’s 2024 Global Findex round, 62% of women held a mobile money account and 20% a bank account, so bank statements miss most women’s records.
• The Sealed-Weights Rubric scores trading record, returning customers, unit numbers and a specific use of the program, at weights published with the call, with 0 to 3 anchors and a list of accepted evidence.
• In 100,000 Italian and Spanish applications, more women on committees left women’s success unchanged; process carries the load.

Criteria that move when the applicant changes

1,247+ founders have applied to AVODA’s programs, and each of those files was judged against criteria somebody chose. The rubric in this piece is the one we would hand any program lead, AVODA’s own panels first.

Banu Ozkazanc-Pan and Susan Clark Muntean, in Entrepreneurial Ecosystems: A Gender Perspective (Cambridge University Press, 2022), argue that the criteria for judging a venture are often “masked by a dominant ideology of meritocracy.” Two studies show the mechanism.

Eric Uhlmann and Geoffrey Cohen, at Yale, asked people to assess an applicant for police chief. The file described either “Michael” or “Michelle,” and the applicant was either streetwise or formally educated. Evaluators rated education as more important to the job when the educated applicant was Michael. They moved the yardstick to fit the applicant, and those who rated themselves most objective moved it furthest. In a third experiment, participants who rated each quality’s importance before seeing the file stopped discriminating.

The second study concerns what gets scored. In 2014 the Canadian Institutes of Health Research split its open competitions into one program that reviewed the proposed science and one that reviewed the caliber of the lead researcher. Holly Witteman and colleagues studied 23,918 applications from 2011 to 2016. Where reviewers scored the science, women’s predicted success trailed men’s by 0.9 percentage points. Where they scored the person, the gap was 4.0 points.

A pitch panel that awards points for “founder quality,” “coachability” or “investment potential” is running the second program. Each asks the reviewer to judge a person after meeting her. The rubric below scores the business and its record, with weights set while every application is still unopened.

Bank statements, sales books and referrals in Kampala

The book’s case study of a Boston accelerator, TechAccelerator, shows sensible criteria sorting by who you know. Of about 1,000 open applications, the director said, the program took two or three. “The bulk comes from referrals.” He looked for an execution track record: a working product, traction, a team, and at lower priority an MIT or Harvard connection. A founder working alone counted against the application. The authors recommend gender-blind applications and, where that fails at interview, interviewers who are “gender aware,” plus implicit bias training for investors and heads of support organizations.

Three of those transfer poorly to Kampala.

Blinding is hard to run. Applications arrive with a phone number, a voice on a screening call and often a referrer’s name, and a panelist may already know the business. A gender-aware design accepts that the panel knows who is applying and builds protections elsewhere.

The evidence behind bias training is thin. A 2019 meta-analysis by Patrick Forscher and colleagues pooled 492 studies of procedures meant to shift implicit bias, with more than 87,000 participants. The procedures moved implicit-bias scores weakly, mostly in single sessions, and “generally produced trivial changes in behavior.” A workshop budget buys more fairness as structure: sealed weights, anchors and a calibration meeting.

The track record also lives in different documents here. In the World Bank’s 2024 Global Findex round for Uganda, 62% of women held a mobile money account against 75% of men, and 20% of women held an account at a bank or similar institution against 33% of men. The bank figure moved sharply between rounds, but mobile money has led among Ugandan women in every round since 2017. A call that demands bank statements asks most women for a record they hold elsewhere. Handsets sort applicants too. In GSMA’s 2024 survey, 85% of Ugandan women and 90% of men owned a mobile phone, while 21% of women and 27% of men owned a smartphone; women running micro-businesses were 32% less likely than male peers to own one. A required pitch video filters on the handset before a judge watches it.

One warning in the same chapter does transfer. The director of a Boston accelerator for founders of color said most investors lacked the market knowledge to assess them, citing Black haircare. A Kampala panel of software people scoring a tailoring workshop has the same gap.

The Sealed-Weights Rubric

Before the call opens, the program lead writes down four weights. That act gives the Sealed-Weights Rubric its name and first rule: weights, anchors and the evidence list are written, dated and published with the call, and stay fixed until the cohort is chosen.

Each criterion carries anchors from 0 to 3. For the trading record: 0 is a pre-revenue idea, 1 is sales described but unrecorded, 2 is sales recorded for under three months, and 3 is the row above. Writing anchors makes the panel agree on what “good” means while every file is closed.

Some things carry a weight of zero: presentation polish, English fluency, a pitch video, the number of co-founders, the school attended and the referrer’s name. “Team” becomes a factual question about who does the work, counting paid casual help and family labor. Constraints such as hours, childcare and travel are asked after selection, so the program plans around them and they stay out of the score.

Two checks keep the sheet honest. A criterion on which every applicant scores the same moves nobody’s rank, so cut it and spread its weight. And applicants will study a published rubric, which is the point, so each criterion ties to something staff can check. For the evidence tiers behind each field, the Said, Shown, Checked grading already does the work; the rubric decides what each tier is worth.

How the panel scores and checks itself

Process carries as much weight as the sheet. Iris Bohnet, Alexandra van Geen and Max Bazerman ran an experiment with American college students choosing a candidate for a math or verbal task. When evaluators judged candidates one at a time, a candidate from the stereotype-favored group was chosen 65% of the time and one from the other group 49%. When they compared two candidates side by side, the gender gap disappeared and they picked the higher performer 54% of the time against 8% for the weaker one.

The panel routine runs in five steps.

1. Score across, then down. Each reviewer scores the trading record for every application, then returning customers for every application, and so on.
2. Use two reviewers with independent scores. Both enter scores before any discussion. The panel discusses only the criteria where two reviewers differ by two points or more.
3. Bring a reader who knows the market. Where the panel lacks someone familiar with a sector, it borrows one for that file, so the business is scored on its own trade.
4. Structure any interview: same questions in the same order, in the founder’s language with an interpreter where needed, used to check the record against the file.
5. Treat panel composition as a separate decision. Manuel Bagues, Mauro Sylos-Labini and Natalia Zinovyeva studied 100,000 professorship applications in Italy and Spain, judged by 8,000 randomly drawn evaluators. More women on a committee left women’s success unchanged, and male evaluators grew less favorable to women once a woman joined. Seat women for their judgment and standing, and let the process carry the fairness.

Calibration brackets the scoring. Before it starts, every reviewer scores the same five past applications alone, and the panel compares results. Any anchor that two reviewers read two points apart gets rewritten that day.

During scoring, staff compare each reviewer’s average with the panel’s on shared files, and a reviewer who runs consistently high or low gets a conversation.

After selection, the report shows women’s share at three steps (applied, shortlisted, admitted) and each criterion’s average by gender. The Score Split set out in Incubators That Select for Confidence Select for Men tells you whether your last intake needed a new rubric; this report tells you whether the new one worked. A year into the cohort, check which criteria tracked completion and continued trading, and adjust the next call’s weights. Weights change between calls and stay sealed during one.

When hiding names cost women their edge

Michael Hiscox and his team at the Australian government’s behavioral economics unit, BETA, tested blind recruitment. In 2017, over 2,100 public servants from 14 agencies shortlisted applicants for a hypothetical senior role. Each was randomly assigned CVs with names or with names removed. With names visible, women were 2.9% more likely to be shortlisted and men 3.2% less likely. The reviewers were already leaning toward women, and the report concluded that de-identification “may frustrate efforts aimed at promoting diversity.”

Carried to Kampala, the trial cuts against any rubric that removes a panel’s discretion. If a panel there already compensates for the barriers women face, sealed weights could remove that compensation and lower the share of women admitted.

That risk is real, and the calibration report is where it would surface: within one intake, at the step where the drop happens. And a lean that lives in panelists’ instincts leaves with them; a program that wants to favor women-led businesses can publish that as a reserved share or tie-break, where funders and applicants can hold it to account. Hiscox’s own foreword points to the same stages this rubric targets: “how interviews are conducted, and how hiring panels are selected and run.”

Hear the small and the great alike

At Horeb, as Deuteronomy 1:9-18 recalls it in the New International Version, Moses tells the people he can no longer carry their disputes alone and appoints judges over thousands, hundreds, fifties and tens. His charge is brief. “Judge fairly.” “Hear both small and great alike.” “Bring me any case too hard for you, and I will hear it.” He told them everything they were to do “at that time,” before the first case arrived. Each device in this rubric is there: one hearing for everyone, a route for the case two judges cannot settle, rules set in advance. Procedure has a weak point, and Uhlmann and Cohen’s evaluators found it: a panel that feels objective can bend a procedure while following it. Moses gives his judges a reason that sits outside the sheet: “judgment belongs to God.” Under that charge the sealed weights hold on the afternoon when bending them would be easy and unseen, and the founder with the thinnest file is owed the hearing her neighbor received.

Frequently Asked Questions

How is a gender-aware rubric different from a women-only call?

A women-only call decides who may apply. A gender-aware rubric governs how each application is judged: weights fixed before reading, criteria tied to checkable records, evidence women commonly hold, and results reported by gender. Its panel can still reward polish over trading.

Should an incubator anonymize applications to reduce gender bias?

It is hard and sometimes counterproductive. In a Kampala intake, phone screens and referrals reveal identity anyway. An Australian government trial found reviewers already favored women, so de-identified files would have lowered women’s shortlisting. Sealed weights, anchored scores and a stage-by-stage gender report do more.

What evidence should a selection panel accept from founders without bank statements?

Accept the records founders keep: mobile money statements, paper sales books, receipt books, supplier invoices, savings group passbooks and customer numbers staff can call. In Uganda’s 2024 Findex round, 62% of women held mobile money accounts and 20% bank accounts.

How should a selection panel score applications to reduce bias?

Have each reviewer score one criterion for every application before the next, with two independent reviewers per file and discussion only where they differ by two points. In a laboratory experiment by Harvard researchers, side-by-side comparison removed the gender gap and favored the stronger performer.

Does adding women to the selection panel remove gender bias?

On its own, the evidence says rarely. A study of 100,000 academic applications in Italy and Spain found more women on committees left women’s success unchanged, and male evaluators became less favorable once a woman joined. Mixed panels bring judgment and standing; the scoring process carries the fairness.

Next Step

Date and sign the four criteria, their weights, the 0 to 3 anchors and the evidence list before your call for applications goes out. Then bring them and five past applications to a strategy call with Joshua, and we will score them together, find the anchors two readers interpret differently and fix them while the call is still a draft. Print the seal date on the call itself, where every applicant can read it and every panelist is bound by it.

Related Reading

• Who Actually Finishes: What 99 Alumni Say About Accelerator Selection
• When a Diagnostic Earns Trust, and When It Becomes a Sales Instrument
• Gender-Lens Investing: Capital First or Capability First?
• Selection vs Treatment: What Accelerators Actually Add

Sources and Evidence

1. Ozkazanc-Pan and Clark Muntean, Entrepreneurial Ecosystems: A Gender Perspective, Cambridge University Press, 2022 (DOI 10.1017/9781009023641): chapter 3 on criteria and meritocracy; chapter 7 on TechAccelerator’s selection and Boston investors; chapter 8 on gender-blind and gender-aware hiring and bias training.
2. Uhlmann and Cohen, “Constructed Criteria: Redefining Merit to Justify Discrimination”, Psychological Science 16(6), 2005: the police chief experiments; commitment to criteria before learning the applicant’s gender eliminated discrimination.
3. Witteman, Hendricks, Straus and Tannenbaum, “Are gender gaps due to evaluations of the applicant or the science?”, The Lancet 393, 2019 (PubMed abstract): 23,918 applications; gaps of 0.9 and 4.0 percentage points.
4. Bohnet, van Geen and Bazerman, “When Performance Trumps Gender Bias: Joint vs. Separate Evaluation”, Management Science 62(5), 2016: separate evaluation 65% against 49%; joint evaluation removes the gender gap.
5. Bagues, Sylos-Labini and Zinovyeva, “Does the Gender Composition of Scientific Committees Matter?”, American Economic Review, April 2017: 100,000 applications, 8,000 evaluators in Italy and Spain.
6. Hiscox and others, “Going blind to see more clearly: unconscious bias in Australian Public Service shortlisting processes”, Behavioural Economics Team of the Australian Government, June 2017: 2,100 public servants, 14 agencies; 2.9% and 3.2%; the foreword’s recommendation.
7. Forscher and others, “A meta-analysis of procedures to change implicit measures”, Journal of Personality and Social Psychology 117(3), 2019 (PubMed abstract): 492 studies, more than 87,000 participants; trivial changes in behavior.
8. World Bank Global Findex, Uganda, mobile money account, women (age 15+), 2014 to 2024: 61.7% in 2024; men 74.6% in the companion series.
9. World Bank Global Findex, Uganda, bank or similar financial institution account, women (age 15+), 2011 to 2024: 19.9% in 2024; men 33.4% in the companion series.
10. GSMA, The Mobile Gender Gap Report 2025: Uganda mobile ownership 85% and 90%, smartphone ownership 21% and 27%; women micro-entrepreneurs 32% less likely to own a smartphone.
11. The New Times, “12 women-led businesses compete ahead of 8th BK Urumuri initiative finale”, 15 November 2024: the seven judging criteria.
12. Deuteronomy 1:9-18 (NIV), Bible Gateway: Moses appoints and charges the judges.