Imagine applying for a loan and it’s rejected by an AI system that can’t explain why. You ask the bank, and they tell you the system assessed your profile and found you weren’t eligible. You push back. The bank blames its technology. The developers say the system did what it was built to do. Nobody can tell you exactly what went wrong, and nobody seems willing to own it.

So who is accountable?

This question worries me more and more as AI shows up in the products we use every day.
Most of the talk about AI is about what it can do, how fast it works and how much money it saves. Far less is said about what happens when it gets things wrong, especially when the mistake affects someone’s income, money, education or access to basic services.

People and organisations design, launch and run AI products. When those products cause harm, the responsibility can’t vanish into an algorithm. For product managers, founders and tech leaders, that makes accountability a product governance problem, and it needs to be settled before the product goes to market.

The problem with blaming the algorithm When normal software fails, you can usually trace it to a bug, an outage or an infrastructure fault. AI is harder. Its output can depend on the training data, how the model behaves, where
it is used and how people read its recommendations.

An AI system can give an answer that sounds right and is false. A credit-scoring model can work better for some customer groups than others. A recruitment tool can screen out qualified people because its training data carried old patterns that should never have been copied.

All this raises questions about whether it was tested properly, whether its limits were understood, and whether the company had safeguards in place. The US National Institute of Standards and Technology’s AI Risk Management Framework sets out four core functions for managing AI risk: govern, map, measure and manage. Put simply, someone has to be accountable, possible harms have to be identified, performance has to be checked, and risks have to be dealt with for as long as the system is in use.

Passing a technical test is only the first step in responsible AI work. A system can work exactly as designed and still be wrong for the job. A model that predicts customer behaviour well on average can do badly for one group. A chatbot can answer most questions correctly and still give dangerous advice in a sensitive case. A fraud system can cut losses while blocking real customers’ payments. If a company only measures overall accuracy, it can miss the people who pay the highest price for its mistakes.

Every AI system can fail, so the people building and launching it need to think through how it might fail, put safeguards in place, and decide what to do when those safeguards aren’t enough.

Who should be accountable?

There’s no one answer for every AI product. It depends on the system, the decisions it makes, who built and deployed it, and the law that applies. Still, a few groups each carry their own share.

The company using the product owns the outcome

A business can’t hand off all its responsibility to an AI vendor just because someone else built the model. If a bank uses AI to make recommendations about customers, it has to understand the system’s limits, judge whether it suits those customers, and have a way to review disputed decisions. Within its legal obligations and the facts of each case, the company is responsible for how it uses the technology.

Developers and providers must know and share their system’s limits

Anyone supplying an AI model or component should test it properly, write down its known limits and flag the risks they can foresee. If it only works well with certain data, conditions or assumptions, the companies using it need to know. A provider shouldn’t claim a system is reliable for every use because it scored well on one narrow benchmark.

Product managers must make risk part of the product decision

Product managers sit between customer needs, business goals, engineering and leadership. They help decide what a product does, which problems matter and what must be true before launch.

That position comes with a duty to ask hard questions. What happens if the system is wrong? Who could get hurt? How accurate does it need to be for this use? Can users challenge a decision that goes against them? Is there a person who can step in when the system behaves strangely? PMs don’t control every technical, legal or operational decision, and they shouldn’t become
the easy scapegoat when a company fails to give them resources or oversight. Their job is to make sure these questions get asked, written down and passed to people who have the power to act.

Leaders and boards must give teams the resources and authority to manage risk

A product team can’t build real safeguards if leadership puts speed to market above everything else. Senior leaders need to set how much risk is acceptable, fund testing and monitoring, and make clear who can delay or stop a launch.

Accountability should follow responsibility and control. It also has to be clear enough that nobody can pass the problem to another department when something goes wrong.

Human oversight takes more than a person in the room

A common answer to AI risk is to say a human will stay in the loop. Having a reviewer
doesn’t make a system safe on its own.

Picture an AI system that screens thousands of loan applications and flags the risky ones. A staff member is meant to review its calls, but they have little time, know little about how the model works, and can’t really overrule it. Oversight exists on paper, and that’s about it.

For oversight to work, reviewers need training, the right information, enough time and the authority to question or reject what the AI recommends. They also need a clear way to escalate unclear or high-risk cases.

In article 14 of the European Union’s AI Act, it demands that sufficiently trained and equipped users are able to recognise the system’s limitations, identify irregularities, ignore or reverse its outputs, and intervene if necessary.

For product teams, that means embedding oversight into the workflow from day one. How much review you need depends on the value of a wrong answer. An AI assistant that suggests a medical action requires far more checking than a tool that suggests slide headings. For low-risk tools, the right amount of protection might be as simple as clear disclosures and a simple way to fix mistakes. If the products affect credit, jobs, health or other serious decisions, you may need stronger testing, human review and a way to appeal.

The danger of testing only for the average user

I’m also concerned about how AI products are tested before launch. Teams often lean on benchmarks, test datasets and overall accuracy to decide a system is ready. Those are useful, but they can hide failures for particular groups. A system can score well overall and still keep failing people who speak a certain language, use a different dialect, or aren’t well represented in the training data.

This matters in Nigeria, where AI products reach people across many languages, regions and income levels. A model tested mostly on English-language data from a narrow group may struggle with local expressions, accents and situations.

Take an AI education product. If it is only tested on standard questions written in formal English, you won’t find out whether it can help students who ask in Nigerian English or other local expressions. In the same way, a customer service bot can look great in a demo and still misread how real customers talk.

So the team has to test the product on the people and conditions it is meant for. That means checking performance across user groups, writing down what is uncertain, testing likely misuse and digging into patterns of failure.

The OECD’s AI Principles stress accountability, transparency, traceability, robustness and safety for the whole life of an AI system. Teams need to understand how a system’s risks are assessed and managed, as well as what it produces.

Testing also has to keep going after launch. As the product, its users, the model and its environment change, so do its performance and risks.

What happens after an AI product causes harm?

Even a carefully built AI product can fail in ways nobody expected, so planning for incidents
matters as much as testing before launch.

Companies need clear steps for taking complaints, investigating failures, telling the right people and fixing problems. When needed, they should be able to switch off a feature, roll back a model update or pause the system while they investigate.
They also need enough records to reconstruct what happened. Depending on the product that could include model versions, inputs, outputs, system settings, test results and any human interventions, all held within privacy and security rules. A bad decision is difficult to investigate without adequate records and you are playing guessing games.

People affected by a decision need a real way to challenge it. If AI helps reject someone’s loan or job application, that person shouldn’t be stuck in an automated loop with no route to a review. A complaints process should lead to an actual investigation, a fix where possible, and changes that make a repeat less likely. For businesses, this protects customer relationships and trust in the product as much as it limits legal risk. A company that is open about failures, fixes its errors and shows it has learned is in a much better position than one that denies blame or hides behind technical jargon.

Nigeria can’t leave AI governance for later Nigerian businesses are trying AI in finance, education, health, commerce and public
services. Governance has to grow at the same pace.

The National Artificial Intelligence Strategy 2024 sets out Nigeria’s goal of using AI for innovation and social and economic growth. Reaching that goal will take as much attention to responsible use as to opportunity.

Nigeria already has laws that cover some AI risks. Section 37 of the Nigeria Data Protection Act 2023 deals with automated decision-making.

AI governance sits alongside data protection, consumer protection, sector rules and other legal duties. Companies should work out which laws apply to their product instead of assuming a general AI policy covers them.

I believe startups will not need an expensive compliance team to begin. They can start with a written risk assessment, clear owners, testing on real users, an incident plan and a set way to raise concerns with leadership. As the product grows and the stakes rise, those controls should grow too. The aim is for responsible AI to become a normal part of building products,
done before any public scandal forces it.

Five questions every product team should answer before launch

Before releasing an AI product, the team and the company should be able to answer these:

1. What can go wrong? Name the likely failures, who they would affect and how serious the harm could be.
2. How have we tested it? Did we test it again after the initial test?
3. Who can step in? Name who monitors the system, reviews disputed outputs, escalates incidents and can switch off unsafe features.
4. Can affected users challenge an outcome? Give them a practical way to complain, get a human review and have mistakes corrected.
5. What happens when it fails? Have an incident plan, keep the right records, tell the people affected, and use what you learn to improve the product.

These questions don’t replace sector standards or legal advice. They are a practical way to start building accountability into the product from day one.

A broader measure of success I’m hopeful about what AI can do. It can help businesses run better, reach more people with
services, and solve problems that used to be too costly or complex to tackle.

Launch speed, user numbers and automation rates matter, but we also need to ask whether the product works reliably, treats people fairly, protects their interests and responds properly when things go wrong.

For product managers, this means a broader idea of what a successful launch looks like.

Finishing development or hitting a strong benchmark doesn’t make a product ready. It is ready when the company understands the risks, has tested it for the job it will do, has safeguards in place, and knows who has to act if those safeguards fail.

The product manager can’t carry this alone. Every organisation that builds, deploys or runs an AI system should share the responsibility, in line with its role and its control over the technology. What PMs can do is make accountability a condition of launch, so the conversation happens before a customer gets hurt.

AI will make mistakes. A product that works has done half its job. The other half falls to the people behind it: seeing those mistakes coming, putting things right, and taking responsibility when it fails.

Adebayo Oluwaseun Abolaji writes.
Adebayo Oluwaseun Abolaji is a technology and product leader with experience spanning software engineering, product management, and digital innovation.