.png){kind=logo}
When a Bank Does Not Build the AI: Governing Third-Party Models and Agents
Artificial intelligence is becoming embedded in banking technology, but banks are not developing or directly operating all of it. Financial institutions increasingly rely on providers whose platforms incorporate predictive models, generative AI, large language models, and agentic capabilities into lending, fraud detection, customer interactions, document processing, and other business processes.These technologies do not present identical risks. A predictive model that produces a fraud score behaves differently from an LLM that summarizes a case or an AI agent that can retrieve information, invoke tools, and initiate actions. The governing question is therefore not simply whether a product “uses AI.” Banks need to understand what the technology can access, what it can produce or influence, what actions it can take, and how those activities could affect customers and banking operations.
A bank may not control how a provider develops its technology, but it remains responsible for determining whether that technology is appropriate for its intended use. Effective governance requires sufficient visibility, accountability, control, and evidence to manage the resulting risk.
Start With Visibility, but Go Beyond a Product Inventory
AI functionality may be embedded within a larger platform or workflow rather than introduced as a standalone application. An inventory is an important starting point, but a list of vendors and use cases is not enough.
For each capability, the institution should understand:
- Its intended business purpose and accountable owner
- The type of model or AI functionality involved
- The data it can access and how that data may be used
- The decisions, recommendations, or actions it can influence
- The degree of autonomy granted to the system
- The role of human review, approval, and override
- The metrics used to evaluate performance
- The evidence retained for monitoring, investigation, and audit
- How material changes are communicated and approved
This information allows the bank to identify not only where AI exists, but where it creates meaningful customer, compliance, operational, or reputational exposure.
Govern According to Risk and Operational Behavior
Third-party AI should not be treated as a single risk category. Oversight should be proportionate to the sensitivity of the data, the materiality of the use case, the potential customer impact, the system’s autonomy, and the institution’s ability to detect and reverse an undesirable outcome.
A tool that helps an employee summarize internal information does not require the same controls as one that contributes to a lending decision. Likewise, an agent that gathers information and recommends an action presents a different risk from one authorized to execute that action.
Banks should ask practical questions:
- Can the system create customer-facing content?
- Can it influence a consequential decision?
- Can it access confidential or regulated information?
- Can it invoke tools, update records, or initiate transactions?
- Must a qualified person approve the result?
- Can an action be stopped or reversed?
- Can the institution reconstruct what occurred?
Existing model-risk, compliance, information-security, and third-party risk disciplines provide an important foundation. They should not, however, be treated as a complete answer for every generative or agentic system.
Expand Vendor Due Diligence
Traditional vendor due diligence remains necessary, but AI-enabled products introduce additional questions.
Banks should understand what role the AI performs, what information supports its outputs, how performance is evaluated, and whether the provider depends on outside model, cloud, or data services. Due diligence should also address:
- Data retention, isolation, and deletion
- Whether bank or customer data may be used for model training
- Known limitations and prohibited uses
- Human-review and override controls
- Access permissions and tool-use restrictions
- Material model, data, or provider changes
- Testing for unsupported or unintended output
- Incident detection, notification, and investigation
- Business continuity, portability, and exit planning
Banks do not need access to every element of a provider’s proprietary technology. They do need enough operational transparency to evaluate its role, manage its risks, and fulfill their own responsibilities.
Require Evidence, Not Just Assurance
Oversight cannot end when a contract is signed or a product is implemented. Models, data sources, configurations, and underlying services can change over time.
Depending on the use case, ongoing evidence may include performance results, exception patterns, human overrides, model or service versions, incidents, and actions initiated by AI agents. For agentic systems, appropriate traceability may also include the instructions provided, evidence retrieved, tools invoked, approvals received, and resulting actions.
Governance should emphasize enterprise-managed AI, controls based on data classification, human accountability for outputs, and additional safeguards for systems that execute multistep tasks or interact with external services. Banks should consider an internal benchmarking approach based on standardized tasks, repeatable execution, and measurable grading rather than relying solely on vendor claims or general-purpose leaderboards.
Put Governance Expectations into the Contract
Contracts should establish what information and cooperation the institution will receive throughout the relationship. Relevant provisions may address:
- Permitted uses of bank and customer data
- Restrictions on using that data for model training
- Material model and provider-change notification
- Performance and control evidence
- Incident reporting and investigation support
- Critical subcontractor dependencies
- Audit and regulatory cooperation
- Human-review requirements
- Continuity and exit obligations
- The right to suspend unsafe or noncompliant functionality
Responsibilities should also be explicit. The provider should explain what technology and safeguards it supplies. The bank should define the approved purpose, operating context, and decisions for which it remains accountable.
What Responsible Technology Providers Should Deliver
Responsible AI cannot stop at a product feature or an accuracy claim. Banks need evidence showing what a capability does, what information it uses, what decisions or actions it can influence, how consequential changes are controlled, and how outcomes can be reviewed.
AI-enabled capabilities should operate inside governed business workflows rather than outside them. That means defined purposes, controlled access to information and tools, accountable ownership, measurable acceptance criteria, versioned change, human oversight for consequential activities, and evidence retained for review and audit.
The objective is not to eliminate innovation or demand disclosure of proprietary technology. It is to ensure that a financial institution can understand the technology’s role, control how it is used, monitor its behavior, and respond effectively when an outcome requires investigation.
As models and agents become more deeply embedded in banking platforms, effective oversight will depend less on whether a capability was built inside or outside the institution. It will depend on whether the bank and its technology provider can demonstrate that the capability is governed, observable, accountable, and worthy of operational trust.
About Author:
Wendy Wilson is Director of Product Management at ARGO, the leading provider of mission-critical technology and analytical-sciences software for the financial services and healthcare industries.