Class 7 · CBSE AI · Strand A — Systems Thinking
AI loan approvals — how bias creeps into credit scoring
How AI credit systems can encode historical discrimination, and what disparate impact means. For Class 7.
What this concept actually says
- AI credit scoring systems can encode and amplify historical discrimination present in training data
- High-stakes automated decisions require auditability — the ability to explain why a specific decision was made
- Disparate impact is when a decision-making system produces systematically different outcomes for different demographic groups, even without using those demographics as inputs
An analogy your child will recognise
Caste-based rental discrimination (historical India)
In many Indian cities, landlords historically refused to rent to certain communities. If an AI learned from those rental records, it would predict that those communities 'don't rent houses in this area' — and therefore not show them listings. The AI didn't 'know' about caste; it learned from data that was already shaped by caste discrimination. The output is discrimination without a discriminating input.
School admission on postal codes
A school that admits students based on postal codes and parental occupation — without ever mentioning class — will produce a student body that is overwhelmingly from wealthy neighbourhoods. The proxy variables do the filtering that the school's rules explicitly don't do. AI loan systems work the same way.
Common misconceptions to watch for
- If an AI doesn't use demographic variables (gender, caste, race), it cannot be discriminatory.
- An AI that is more accurate overall is necessarily fairer — accuracy and fairness across groups are independent properties.
Key facts in one breath
- Disparate impact: a facially neutral policy or algorithm produces significantly different outcomes across demographic groups — this is legally and ethically significant even without discriminatory intent.
- In 2019, a US healthcare AI was found to systematically underestimate the health needs of Black patients because it used healthcare spending as a proxy for health need — lower spending (due to systemic discrimination) was incorrectly read as lower need.
- Explainability in credit decisions is legally mandated in many countries — applicants must be told why they were denied.
- Fairness in AI can be defined in multiple mathematically conflicting ways — it is impossible to satisfy all fairness criteria simultaneously in all cases.
How Dhee Learning teaches this — the 3-stage question loop
Every Dhee Learning session for this concept follows three stages. We share the questions Dhee actually asks, so you can hear what a session sounds like.
Stage 1 — Surface
Imagine a bank uses an AI to decide who gets a home loan. The AI was trained on 10 years of the bank's past loan decisions. If the bank had historically been biased against lending to women, what would you expect the AI to learn?
Rote answer
"The AI would be biased against women too."
Understood
"The AI would learn that women are 'riskier borrowers' — not because they actually are, but because the historical data reflects the bank's past discrimination, not women's actual repayment rates. The AI would replicate and potentially amplify the discrimination because it treats past decisions as ground truth."
Stage 2 — Reasoning
An AI loan system doesn't use gender or caste as inputs. Yet women from scheduled castes are approved at rates 40% lower than men from upper castes with similar credit profiles. How is this possible, and what is it called?
Follow-up Dhee may use: If you were a regulator reviewing this AI, what data would you ask the bank to show you to determine whether disparate impact is occurring?
Stage 3 — Application
You are advising a microfinance company that wants to use AI to approve small business loans in rural India. Write a five-point design checklist that ensures the system is both accurate and fair. For each point, explain what problem it prevents.
Misconception Dhee watches for: Child lists technical accuracy metrics (AUC, precision) as fairness measures — accuracy and fairness are distinct and can trade off against each other.
Related concepts
Want your child to actually understand this?
Dhee turns this concept into a 15-minute spoken session — asking, listening, and probing — so your child builds the idea themselves.
Frequently asked questions
What is case study — ai loan approvals — explained for kids? +
How AI credit systems can encode historical discrimination, and what disparate impact means. For Class 7.
What's the most common mistake children make about this concept? +
If an AI doesn't use demographic variables (gender, caste, race), it cannot be discriminatory.
How does Dhee Learning teach this in a Class 7 session? +
Dhee opens with a question — for example: "Imagine a bank uses an AI to decide who gets a home loan. The AI was trained on 10 years of the bank's past loan decisions. If the bank had historically been biased against lending to women, what would you expect the AI to learn?" — listens to your child's answer, then probes the reasoning behind it. The session ends when the child can apply the idea to a brand-new situation, not just recall it.