A new study published in Science Advances has added an important layer to our understanding of AI bias. Researchers found that large language models (LLMs) — the systems behind many of our everyday AI tools — evaluate identical text differently depending solely on who they believe wrote it.
The content stays the same. The reasoning stays the same. But the evaluation shifts when an identity is attached.
This discovery has major implications for organisations beginning to use AI in hiring, content moderation, performance evaluation, policy consultation, service triage, and employee experience.
And interestingly, it connects directly to something many of us already understand well: blind hiring.
What the Study Found
Researchers asked several leading LLMs to evaluate argumentative statements across topics such as public health, geopolitical issues, and social policy.
They tested two conditions:
- Anonymous text — the model sees the content but no author information.
- Attributed text — the exact same content is paired with an author identity (e.g., a nationality or a note saying it was written by an AI system).
The results were striking in their consistency:
- Under anonymous conditions, the models produced stable and aligned evaluations across systems.
- Once an author identity was added, evaluations changed — sometimes subtly, sometimes significantly.
A few patterns stood out:
- Text attributed to a Chinese author generally received lower ratings, highlighting a nationality-related bias.
- Identical text labelled as AI-generated was scored less positively than when labelled “human-written.”
- These effects appeared across multiple models from different developers.
This is not an isolated quirk. It’s a pattern.
Why This Matters for Organisations
AI is rapidly being integrated into organisational workflows — often in roles that require some level of judgement. For example:
- Screening CVs or shortlisting applications
- Prioritising customer or staff complaints
- Moderating content or submissions
- Summarising consultations or community feedback
- Supporting academic, grant, or internal review processes
- Assessing performance or identifying risk signals
If an AI model changes its judgement based on irrelevant identity cues, several risks emerge:
1. Individuals from certain backgrounds may be unintentionally disadvantaged.
Bias does not have to be malicious to be harmful.
2. Decision-making becomes inconsistent and less transparent.
A consistent process can’t be fair if the AI’s evaluation shifts based on metadata alone.
3. Organisations may unknowingly embed structural inequities into automated systems.
Even well-intentioned AI use can replicate unequal patterns from the past.
4. Trust in AI systems — internally and externally — can erode.
People must believe that an AI-supported process is fair, or they will reject the outcome.
A Useful Parallel: Blind Hiring
The results from this study parallel something we have long known in DEI and organisational psychology: Removing identity cues can significantly reduce bias.
Blind hiring practices — such as redacting names, gender markers, and other identifiable details — were designed to give all candidates an equitable starting point by focusing on capability rather than assumption.
This research suggests AI needs the same treatment.
When models don’t see the author identity, their evaluations are far more consistent. When metadata is revealed, bias reappears.
Blind hiring isn’t about hiding identity forever — it’s about preventing identity from influencing early, high-impact decisions.
Applying the same principle to AI may be one of the most practical fairness interventions available to us right now.
How to Use These Insights Responsibly
To support fair and inclusive AI adoption, organisations can take several evidence-informed steps:
1. Avoid using LLMs as autonomous evaluators.
AI should assist human judgement, not replace it in high-stakes assessments.
2. Remove author or source metadata before AI evaluation.
Let the model focus on content, not identity.
3. Conduct deliberate bias audits.
Test how models respond to nationality, language style, gender cues, and AI-vs-human labels.
4. Maintain strong human-in-the-loop governance.
Particularly in hiring, promotions, complaints handling, or disciplinary processes.
5. Build transparency into your AI governance.
Document what information is provided to AI systems and why.
These steps are not complicated — but they require intention, alignment, and leadership.
The Bigger Picture: Designing AI That Supports Inclusion
AI is shaped by human data and human patterns. It is not — and cannot be — inherently neutral.
But with thoughtful design, clear principles, and the right governance, AI can be part of a more inclusive, equitable organisational future.
Just as blind hiring reshaped recruitment by removing information that shouldn’t influence a decision, metadata management may become a core pillar of ethical and inclusive AI design.
Related Reading:
Is AI Set to Become the Most Powerful Discriminator of Our Time?
Why Has Women’s Visibility Collapsed on LinkedIn?
Stereobots: When Chatbots Get Typecast — and How We Can Recode Gender in AI
The Intersection of AI and DEI
AI and Organisational Culture: Lessons from Deloitte on Trust, Transparency, and the Human Factor
Guarding Against AI Bias in Talent Management: Building Fairer, Smarter Workplaces
Integrating AI into Your Next DEI Strategy
An Invitation
If you’re interested in building AI systems — and AI policy — that actively support fairness, inclusion, and accountability, **join us at **ada.ai and help shape the future of inclusive AI.
[
](https://www.linkedin.com/showcase/ada%E2%80%94ai/?viewAsMember=true)