In Depth | October 2026 Special
AI: Does the Board Understand the Business It's Agreeing to Become?
Artificial intelligence presents enormous opportunities for financial services firms. Faster decisions, improved customer experiences, reduced operating costs and more efficient processes are all attractive propositions. I expect Boards to encourage management to explore them.
But I wonder how many Boards are genuinely considering the longer-term implications of the businesses they are creating.
Imagine an investment proposal promising a 40% reduction in operational headcount or outsourcing costs through the adoption of AI.
The financial case looks compelling. Management has prepared detailed forecasts, identified the efficiencies and outlined an implementation plan. Risk, Compliance and Technology have reviewed the proposal. Internal Audit has reviewed the final gated plans, with no material concerns.
There may be every reason to proceed...
Before approving it, I'd want to understand what the business expects to gain, what it might be giving up and whether management has properly considered the consequences if its assumptions prove wrong.
The decision could fundamentally change how the organisation operates, the skills it retains, its reliance on third parties and its ability to serve customers when something goes wrong.
Does management really understand AI?
I'm not suggesting that every executive or Board member needs to become an AI specialist.
I would, however, expect management to demonstrate sufficient knowledge of the technology it proposes to introduce, particularly where its use could materially affect customers or change the operating model.
That is quite a difficult expectation to satisfy.
AI is developing rapidly. Even those with considerable technical expertise cannot anticipate every circumstance in which a system might behave unexpectedly.
An AI specialist may understand the technology exceptionally well but have limited knowledge of a firm's regulatory obligations, customer base or operational processes. Equally, an experienced financial services executive may understand the business but lack the technical knowledge to challenge the assumptions underpinning an AI proposal.
So how has management brought those skills together?
Who has assessed the technology's limitations? Who understands the implications for existing processes and controls? How have knowledge gaps been addressed?
External expertise can be valuable, and firms should seek it where appropriate. But appointing a specialist does not remove the need for management to understand the technology it will ultimately be responsible for operating.
The FCA's October 2026 publication on AI good and poor practice identifies weaknesses in governance, accountability and the involvement of control functions. It also highlights examples of firms bringing business, technical and risk expertise together before deployment.
As a Board member, I would want management to explain, in its own words, how the technology will operate within the business, where it could fail and what arrangements are in place to manage those risks.
A supplier presentation or external assurance report would not be enough on its own.
How do we know the outputs are appropriate?
Take an AI-enabled credit application process.
The system might process applications more quickly, produce consistent recommendations and demonstrate impressive accuracy during testing.
But how was that testing conducted?
Have outputs been assessed against the firm's credit risk policies and affordability requirements? Have the implications for complaints handling, vulnerable customers and wider customer outcomes been considered?
How does the system respond when information is incomplete, a customer's circumstances are unusual or several policies need to be considered together?
An AI system could perform exactly as designed and still produce an inappropriate outcome for a particular customer.
Management should be able to demonstrate that testing covers realistic scenarios, including those where an incorrect output could have significant consequences.
I'd also want to understand how the firm has translated its risk appetite into measurable tolerances for AI performance.
A reported 98% accuracy rate may sound reassuring. Whether it is acceptable depends on what the system is doing, the nature of the remaining errors and the potential consequences for customers.
There is also the question of who determines whether an output is correct.
For some applications, that may be straightforward. For others, particularly those involving judgement, vulnerability or competing policy considerations, establishing an appropriate outcome may be considerably more complex.
The FCA's recent findings highlight testing across different consumer characteristics and circumstances, managing bias and monitoring outcomes after deployment. Firms must also consider their existing obligations under the Consumer Duty where applicable.
Management should own the testing, understand the results and be able to explain where limitations remain.
And I would want to see what happens after implementation.
Testing before deployment cannot provide indefinite assurance over a system whose performance may change as models, data and business circumstances evolve.
Management needs to know what it is monitoring, what would indicate deteriorating performance and when intervention is necessary.
I'd be particularly interested in the circumstances that would cause management to suspend the system.
If those circumstances haven't been defined, I'm not sure how the Board can be confident that the business will recognise when it needs to act.
What are we giving up in return for savings?
Let's return to the proposed 40% reduction in operational costs.
It may be an entirely sensible commercial decision. Some existing processes are inefficient, expensive or unnecessarily dependent on manual intervention. AI could improve consistency and allow employees to focus on more valuable activities.
But where the financial benefits depend on reducing headcount or outsourcing arrangements, management needs to explain which operational capabilities the business intends to remove.
Experienced employees possess knowledge that isn't always documented in policies or procedures. You'll find it everywhere, from the senior leadership team to the customer service floor.
They recognise unusual circumstances, understand how processes interact and know when something doesn't look right. They may also be the people who keep services operating when systems or established procedures fail.
Some of that expertise may become less necessary as technology improves. Other capabilities will remain essential.
Before making substantial workforce reductions, I'd expect management to consider parallel running, staged implementation, retaining critical expertise and testing alternative operating arrangements.
I have a wider concern.
If financial services firms increasingly replace particular operational roles with AI, we may eventually find that some skills become much harder to recruit or recover.
And who informs the model?
An individual firm might reasonably assume it could rebuild its operational capacity if required. But what happens if the wider market has undergone the same transformation?
I don't think we can assume that expertise removed today will always be readily available tomorrow.
The financial case should recognise the cost of retaining sufficient capability, not simply the savings associated with removing it.
And when something goes wrong?
This is where I would spend the most time challenging the proposal.
If AI becomes central to customer servicing, credit decisions or complaints handling, the firm needs to understand how those services will continue during a disruption.
Let's say an AI system supporting an important business service becomes unavailable.
How long can the firm continue operating before the disruption exceeds its impact tolerance? And what happens to customers in the meantime?
Can they still contact the business? Can applications be assessed? Can complaints be investigated and resolved? Are there sufficient trained employees to operate alternative processes?
And have those arrangements actually been tested?
I would be lying if I said reputational damage wasn't one of my biggest concerns. But before we even get there, we have customers who may be unable to access services they depend on.
A documented business continuity plan is useful. I'd want evidence that the arrangements could operate under realistic conditions, particularly after the proposed reductions in operational capacity.
I'd also want management to consider what happens when the technology doesn't fail completely.
An obvious outage is likely to trigger an incident response.
A system that continues operating while producing incorrect or inappropriate outputs may be much harder to identify. Customers could be affected before the business recognises that something is wrong.
How does management prepare for that scenario?
Monitoring and escalation arrangements need to cover both system availability and the reliability of the decisions or recommendations being produced.
Third-party dependencies add further complexity.
AI-enabled processes may rely on external technology providers, cloud infrastructure, credit reference agencies and outsourced operational teams. The firm needs to understand those dependencies, the information being shared and the arrangements for protecting customer data.
Under the FCA's operational resilience framework, firms within scope must identify their important business services, set impact tolerances and undertake appropriate mapping and testing. AI dependencies need to be reflected in those arrangements.
The FCA's latest AI findings also identify poor practice where firms relied on untested supplier assurances or failed to establish viable fallback and exit arrangements.
However unlikely management considers a disruption to be, I would be concerned if the financial benefits were presented without evidence that the business could continue serving customers when the technology or one of its critical dependencies failed.
Who is accountable?
AI doesn't remove the firm's regulatory obligations. Nor does reliance on a technology provider transfer responsibility for customer outcomes away from the business.
The FCA's existing regulatory framework continues to apply, including relevant governance and systems and controls requirements, the Consumer Duty and the Senior Managers and Certification Regime.
Identifying an accountable executive is a starting point.
Does that individual have sufficient knowledge of the technology and its risks? Do they receive information that enables them to recognise emerging problems? Can they intervene when necessary?
What responsibilities have been delegated, to whom, and with what skills and experience?
The FCA's recent findings include examples of senior managers delegating AI development to technical teams without sufficient understanding or challenge.
I'd also want to understand how human oversight works in practice.
A human-in-the-loop arrangement may provide an important safeguard, but only if the reviewer has the knowledge, information and authority to challenge an output.
If employees routinely accept recommendations they don't understand, the presence of a human reviewer may provide less protection than management assumes.
How are reviewers trained? Are overrides monitored? Can the business identify patterns of inappropriate reliance on AI-generated recommendations?
Responsibility needs to remain clear throughout the process, including when customers require remediation.
What should independent assurance tell the Board?
For a material AI deployment, particularly one affecting customer outcomes or fundamentally changing the operating model, I would expect management to consider independent assurance.
That doesn't necessarily mean appointing a Big Four firm.
The relevant considerations are expertise, independence, understanding of the business and the quality of the work undertaken. A familiar name on an assurance report is no substitute for those things.
But I would be uncomfortable if management relied on an external report without being able to explain the testing undertaken or the conclusions reached.
Management should have completed its own assessment, understood the results and identified the limitations before asking someone else to provide assurance.
Independent reviewers can then examine the methodology, challenge assumptions, assess controls and identify areas requiring further work.
The Board should be able to question both management and the assurance provider about the findings, particularly any limitations in scope, unvalidated assumptions or gaps in the evidence.
A positive assurance opinion is useful. It doesn't necessarily mean every material risk has been addressed.
Independent assurance should test management's understanding, not compensate for its absence.
What would I expect from the Board?
Despite all of my challenges, I don't think effective Board oversight means making AI adoption unnecessarily difficult.
The opportunities are substantial, and firms that fail to explore them may expose themselves to different commercial and strategic risks.
I continually remind firms that every day and every decision in financial services involves risk, whether we recognise it or not.
If we don't take managed risk, we don't exist.
Nor should every AI application receive the same level of scrutiny.
A relatively low-risk internal productivity tool warrants a different approach from technology influencing credit decisions, customer vulnerability assessments or a significant restructuring of the operating model.
Where the consequences are material, I'd expect the Board to look beyond the financial projections and implementation timetable.
I want management to demonstrate that it understands the technology, has tested its outputs against the firm's obligations and risk appetite, and has credible arrangements for managing disruption and unexpected outcomes.
The Board should also consider whether implementation needs to be staged, whether critical expertise should be retained and whether the anticipated savings remain achievable once appropriate safeguards are included.
That challenge may result in a revised proposal.
Equally, it may give the Board greater confidence to approve the original recommendation.
Either can represent effective oversight.
The business we're agreeing to become
The FCA's recent AI publications reinforce that firms do not need to wait for an entirely new regulatory framework before considering how AI affects their responsibilities.
Existing obligations remain applicable. Firms need to demonstrate that their governance and controls continue to work as AI becomes more embedded in their operations.
I am enthusiastic about the opportunities AI presents. At this point, it practically manages my household! I can see why Boards and management teams are keen to move quickly.
But when a proposal could fundamentally change how an organisation operates, I want the Board to consider the longer-term consequences alongside the immediate commercial benefits.
The skills the business retains. Its reliance on technology and third parties. Its ability to exercise judgement. The arrangements for identifying inappropriate customer outcomes. And its capacity to continue operating when things don't go to plan.
All of these belong in the business case.
So, as a Board member, before approving a significant AI-led transformation, I would want to know:
Does the Board understand the business it's agreeing to become?
And has management demonstrated that it knows how to operate that business when the assumptions underpinning the investment no longer hold?
Six questions I'd ask before approving an AI-led transformation
1. Does management genuinely understand what it's proposing?
Can it explain the technology, its limitations and the gaps in its own expertise without relying entirely on a supplier?
2. How do we know the outputs are appropriate?
What testing demonstrates alignment with our policies, risk appetite and obligations to customers, including those who are vulnerable?
3. What capabilities are we giving up?
If the savings depend on removing people or suppliers, what knowledge and operational capacity will we lose, and could we recover them?
4. What happens when the system fails, or worse, doesn't realise it's failing?
How will we detect inappropriate outputs, continue serving customers and operate within relevant impact tolerances?
5. Who is accountable, and can they genuinely intervene?
Do the people responsible have the knowledge, authority and information to challenge, suspend or override the system?
6. What evidence should give this Board confidence?
What has management tested itself, what has been independently challenged, and which assumptions remain unproven?
Regulatory references
FCA, AI in financial services: good practice and areas for improvement, 19 October 2026.
FCA, Building the future of finance in the age of AI, 19 October 2026.
FCA, FG22/5: Final non-Handbook Guidance for firms on the Consumer Duty.
FCA, Operational resilience guidance and relevant Handbook requirements (SYSC 15A).
This publication is provided for general information only and does not constitute legal or regulatory advice.