When AI Makes Financial Decisions: Who Is Responsible When Things Go Wrong?

Why It Matters 

Artificial intelligence is transforming banking, from approving loans to detecting fraud. But as financial institutions rely more heavily on AI, they also face growing legal and ethical risks that cannot simply be outsourced to technology. 

Key Takeaways 

• AI can unintentionally reinforce discrimination, even when sensitive information such as gender is excluded from its data. 

• Banks must ensure AI systems comply with privacy laws but remain transparent enough for decisions to be explained and challenged. 

• While AI offers enormous benefits in areas such as fraud detection and regulatory compliance, human oversight remains essential. 

AI Is Changing Banking, but the Rules Are Catching Up 

Artificial intelligence has become an integral part of modern banking. Financial institutions now use AI to process documents, assess creditworthiness, detect fraud and personalise financial products. As these technologies become more capable, they also make decisions that can have significant consequences for individuals and businesses. 

Yet the legal frameworks governing AI have struggled to keep pace. For example, the European Union's Artificial Intelligence Act defines AI in ways that reflect older technologies rather than today's machine learning and generative AI systems. As AI continues to evolve rapidly, regulators, courts and businesses may find it increasingly difficult to determine exactly which technologies fall within existing legislation. 

Rather than relying solely on new AI-specific laws, banks should recognise that many existing legal obligations already apply. Responsibilities relating to governance, risk management, accountability and consumer protection remain relevant regardless of how advanced an AI system becomes. 

Bias Doesn't Disappear Just Because Sensitive Data Is Removed 

One of the greatest risks of AI in finance is that automated systems can unintentionally discriminate against certain groups of people. 

A well-known example emerged in 2019, when customers questioned why the Apple Card appeared to offer substantially higher credit limits to husbands than to their wives, even when the women reportedly had stronger credit histories. Although Goldman Sachs, the issuing bank, stated that gender was not used as an input in its algorithm, this did not necessarily eliminate the possibility of bias. 

AI systems often identify patterns using many different data points. Even if gender is excluded, other information, such as employment history or income patterns, may closely correlate with gender and effectively act as substitutes. As a result, removing one variable alone does not guarantee fair outcomes. 

This highlights a broader challenge for financial institutions. AI models can inherit existing inequalities from historical data or develop new forms of bias that remain difficult to detect. Without careful monitoring and testing, banks risk making unfair lending decisions while being unable to explain how those outcomes were reached. 

Privacy and Transparency Matter as Much as Accuracy 

The growing use of AI also raises important questions about customer privacy. 

Banks frequently rely on large volumes of customer information to develop personalised services or recommend new financial products. However, feeding personal data into AI systems may create legal risks, particularly when external technology providers are involved. In many jurisdictions, organisations must ensure customers understand how their information will be used and obtain appropriate consent before data is processed for new purposes. 

Transparency is equally important. Customers, regulators and even banks themselves need to understand how AI reaches its conclusions, particularly when those decisions affect access to credit or other financial services.