Auditing AI candidate scoring for proxy bias

Auditing AI candidate scoring for proxy bias comes down to three habits. Log every scoring decision (the inputs the model saw, the score it produced, the reason it gave and what a human did next), inspect those inputs for proxy variables such as postcode, school name or employment gaps that quietly encode protected traits, and run periodic outcome checks comparing how different groups fare at each pipeline stage. Under the EU AI Act, hiring AI is high-risk, so this is no longer optional hygiene; it is a documented duty.

Why does a fair-looking model still discriminate?

Scoring models almost never see protected attributes directly. No serious vendor feeds ethnicity or age into a ranker. The problem is that any model shaped by historical hiring data learns whatever correlates with past decisions, and plenty of innocent-looking fields correlate with protected traits. If your past hires skewed toward three universities and unbroken CVs, the model will reward those signals, and the people it quietly penalises are not randomly distributed.

This is proxy bias: discrimination laundered through legitimate-looking variables. It survives the obvious fix, removing the protected attribute, because the attribute was never there to begin with. It is also why “our model does not consider gender” is a marketing sentence, not an audit finding. The only way to know what a scorer actually does is to watch what it does, decision by decision, over time.

Which variables act as proxies?

The usual suspects, and what they can smuggle in:

Variable	What it can encode	Why it slips through
Postcode or district	Ethnicity, socioeconomic background	Reads as commute practicality
School or university name	Class, nationality, age	Reads as education quality
Employment gaps	Parenthood, illness, disability, caring duties	Reads as commitment
Graduation year, career length	Age	Reads as seniority
Name and language patterns	Ethnicity, gender, migration background	Reads as writing quality
Hobbies and memberships	Gender, class, religion	Reads as culture fit

None of these fields is forbidden everywhere; commute distance can matter for an on-site shift role. But each one deserves scrutiny: if the model can see it, you should be able to say why, in writing. The same proxies leak in upstream through requirements text, which is why it pays to check whether your job description is generic or biased before any scorer runs.

What should you log for every scoring decision?

The test is simple: could you reconstruct, twelve months from now, why this candidate got this score and what happened because of it? That requires five things per decision:

• Inputs, snapshotted exactly as the model saw them, not a pointer to the live record, which will have changed by the time anyone asks.
• Output: the score and the rank at the moment of scoring.
• Reason: the model’s stated reasoning. A bare number is unauditable; if your tool cannot explain a score, that is itself a finding. How these scores get produced in the first place is covered in candidate scoring in an ATS, explained.
• Version: which model, prompt or configuration produced it. Vendors update models; your January scores and your June scores may come from different systems.
• Human action: who saw the score and whether they followed it, overrode it or ignored it.

That last item is your strongest evidence that the system supports decisions rather than makes them, which is precisely the line GDPR Article 22 and the AI Act care about. Without the trail, you cannot answer a candidate complaint, cannot investigate a skew you suspect, and cannot prove oversight existed.

How do you actually run an outcome check?

Pick a cadence, quarterly for most teams, and compare pass-through rates across groups at each stage: scored, screened, interviewed, offered. You are looking for a group that enters in normal proportions and thins out at one specific gate, especially the scoring gate. The four-fifths rule from US practice (flag any group whose selection rate falls below 80 percent of the best-performing group’s) is blunt, but it is a serviceable tripwire.

The honest complication: under GDPR you mostly do not hold protected-attribute data, and you should not start collecting it casually just to audit. Practical options include aggregate analysis on what you can lawfully infer, voluntary self-reported diversity data collected separately and with clear consent, and attribute-free checks such as whether employment gaps or graduation years correlate with score independently of skills. Document the method you chose and its limits; a documented imperfect check beats an undocumented perfect intention.

When you find a skew, resist the quiet fix. Investigate which input drives it, change the input set or the weighting, and write down what you changed and why. An unrecorded correction looks like a cover-up the day a regulator or claimant asks.

What does the EU AI Act expect you to document?

Recruitment AI sits on the Act’s high-risk list, and using a vendor’s scoring tool makes you a deployer with duties of your own, not a bystander. In practical terms:

• Operate the system according to the vendor’s instructions for use, which the vendor is obliged to give you.
• Assign human oversight to a named, competent person with real authority to override outputs.
• Keep the logs the system generates under your control for at least six months.
• Inform candidates that AI is in use, and inform workers and their representatives where it touches them.
• Monitor performance in use and report serious issues back to the vendor.

Most of the documentation duty is assembly: the vendor’s technical documentation on one side, your audit trail and outcome checks on the other. The trail described above is not extra work on top of compliance; it is the compliance. The wider legal picture is in our guide to the EU AI Act and recruitment software, and the data-protection half lives in what to look for in a GDPR-compliant ATS.

Which questions should you ask a vendor?

Five that separate auditable systems from black boxes:

1. Can we export, per candidate, the inputs, score, reason and model version for every scoring decision? In what format, and how far back?
2. Which fields does the model actually use, and can we exclude specific ones, such as postcode, school names or dates?
3. What bias testing do you run, on what data, and will you put the results in writing?
4. When you update the model, do our scores change, and are we notified before they do?
5. What do you provide for our AI Act deployer duties: instructions for use, logging, technical documentation?

A vendor that answers crisply has built for this; one that pivots to accuracy claims has told you something too. We keep a longer list of ATS demo questions that expose fake AI, and the bias questions belong on it.

Where does Recruitifly fit?

Recruitifly’s assistant, Fly, scores and ranks candidates against a job and shows its reasoning alongside the rank, so the reason column of your audit trail is filled in rather than reverse-engineered. More importantly, scoring never silently becomes action: every write in Recruitifly is propose-then-confirm, so a recruiter approves each change before it happens. That puts a named human between every score and every consequence, which is the oversight posture the AI Act expects, by design rather than by policy memo. The platform is built in the EU with EU data residency, and the GDPR tooling (retention windows, consent tracking, deletion requests, suppression lists) covers the data-protection side of the same audit. Teams that want the duties operationalised further can add the Compliance Engine.

We are in private beta. If you are working out how to make AI scoring defensible rather than merely impressive, talk to us and bring your hardest vendor question; we would rather answer it now than in your audit file later.