Frameworks only matter if they change real decisions. In this final chapter we follow one realistic AI system, from the first idea to life after launch, and watch governance shape it at every step. Then we turn the whole course into a practical checklist you can use to make your own NeuraViz AI Challenge proposal stronger.
By the end of this chapter you can
- Apply the lifecycle gates from Chapter 5 to a concrete, high-impact use case.
- Design human oversight, contestability and transparency for an AI decision.
- Choose evaluation, fairness and monitoring measures for a real deployment.
- Use a ten-point governance checklist to strengthen your own AI proposal.
The case (fictional, but realistic)
"KhetCredit": faster small loans for farmers
A cooperative bank in north India wants to approve small crop loans in days instead of weeks. Its idea: an AI system that combines a farmer's repayment history, satellite data on crop health and a voice-based application in the farmer's own language, and recommends whether to approve a loan and for how much. Thousands of families' livelihoods depend on these decisions, so getting governance right is essential.
Swipe sideways to see the full diagram.
Step 1 · Frame the problem honestly
The team starts with the problem, not the technology. Delays come from manual document checks and field visits. AI could speed up the assessment, but the bank agrees that the goal is faster, fairer access to credit, not simply fewer staff. It records success measures: decision time, approval rates for first-time borrowers, repayment rates and complaint levels. It also records what the system must not do: it will never make a final rejection on its own.
Step 2 · Assess the impact and classify the risk
An impact assessment lists who is affected: applicants, their families, loan officers, the bank and, indirectly, village economies. It identifies the most serious possible harms:
- Wrongful rejection of creditworthy farmers, pushing them towards informal moneylenders.
- Bias against tenant farmers and women, who often lack land titles in their own names and so have thinner records.
- Exclusion of people without smartphones or who speak a less common dialect.
- Privacy harm from collecting more personal and location data than needed.
Because the system influences access to credit, the team classifies it as high risk. (It would also be high-risk under the EU AI Act, and the RBI's guidance on responsible AI in finance applies.) That decision triggers the strongest set of controls and approval by the bank's AI governance council.
Step 3 · Govern the data
First draft
- Collect everything: contacts, location history, caste, social media.
- Train on ten years of past approvals.
- Notice only in English.
After governance review
- Only data needed for repayment ability; no caste, religion or contacts.
- Past approvals reviewed for historical bias before training.
- Consent notice read aloud in the farmer's language (DPDP).
The team also writes a datasheet for its data. It discovers that tenant farmers make up only 4% of historical borrowers, because they were rarely approved in the past. Training on that history would teach the model to repeat the exclusion, so they add alternative signals (for example, satellite evidence of consistent cultivation) and collect a targeted, consented pilot dataset.
Step 4 · Build and test for fairness, not just accuracy
The team chooses a model that can give reason codes (such as "short repayment history" or "low crop-health score this season") over a slightly more accurate black box, because applicants and officers need to understand decisions. Testing goes well beyond overall accuracy:
| Test | Finding | Action |
|---|---|---|
| Approval rate by gender | Women approved 11 points less often at equal repayment records | Removed a proxy feature linked to land title; gap fell to 2 points |
| Accuracy by district | Hill districts scored worse because of cloud cover in satellite images | Added a manual field-check route when image quality is low |
| Voice interface by dialect | Speech errors doubled for two dialects | More training audio and a "speak to a person" option |
| Drought-year stress test | Model over-rejected everyone after a bad monsoon | Policy rule: no automatic downgrades during declared drought |
| Red teaming | Edited crop photos could inflate scores | Image checks and random field verification |
Step 5 · Design human oversight and the right to challenge
Oversight has to be designed into the decision flow, not just written into a policy:
Swipe sideways to see the full diagram.
Two design details make oversight meaningful rather than a rubber stamp. Loan officers are trained on the model's limits, and their dashboard shows the reason codes, not just a score, so they can disagree with evidence. The bank also monitors automation bias: if officers agree with the AI 99.8% of the time, that is a warning sign that they have stopped thinking, not proof that the model is perfect.
Step 6 · Be transparent, launch carefully, keep watching
- Transparency: applicants are told, in their language, that AI assists the decision, what information it uses and how to appeal. A model card and system card are kept on file.
- Approval gate: the governance council reviews an evidence pack (impact assessment, datasheet, test results, oversight design, monitoring plan) and approves a limited launch with conditions.
- Canary launch: two districts for one season, compared against districts using the old process.
- Monitoring: a monthly dashboard tracks decision time, approval and repayment by gender, landholding and district, appeal outcomes, drift in satellite data and complaints, each with an owner and alert threshold.
- Review: an independent audit after the first season, and a decision to expand, adjust or stop.
Apply it: strengthen your challenge proposal
You do not need a governance council to think like one. A proposal that answers these ten questions shows judges and future partners that your idea is not only clever, but ready for the real world:
| # | Question to answer in your proposal | Course link |
|---|---|---|
| 1 | Problem and people: who has this problem, how big is it, and who benefits? | Ch. 8 |
| 2 | Why AI: why is AI better here than a simpler rule-based or manual solution? | Ch. 1–3 |
| 3 | How it works: which approach (model type, RAG, agent, vision, speech) and why? | Ch. 1–3 |
| 4 | Data: where does it come from, is it lawful and consented, and who is missing from it? | Ch. 5, 7 |
| 5 | Risks: what are the top three things that could go wrong, and for whom? | Ch. 6 |
| 6 | Fairness: how will you test that it works for every group of users? | Ch. 4 |
| 7 | Human oversight: where do people stay in control, and how can users challenge outcomes? | Ch. 5, 8 |
| 8 | Transparency: how will users know AI is involved and understand its outputs? | Ch. 7 |
| 9 | Measuring success: which metrics, and why those (recall vs precision, impact)? | Ch. 4 |
| 10 | Scale and sustainability: how will you monitor, update and fund it after launch? | Ch. 4, 6 |
Make it concrete
Instead of "our AI will be fair", write: "We will measure accuracy separately for male and female farmers and for each district, and we will not launch if any group's accuracy is more than 5 points below the average." Specific, measurable commitments are what make governance credible.
Key takeaways
- Governance shapes the product: it changed KhetCredit's data, model choice, decision flow and launch plan, and made it better.
- High-impact AI needs impact assessment, data governance, fairness testing, meaningful human oversight, transparency, a careful launch and continuous monitoring.
- Watch for automation bias: oversight only works if people can, and do, disagree with the AI.
- Use the ten-question checklist to turn your challenge idea into a credible, trustworthy proposal.