Blog

Third-Party and Generative AI Vendor Risk Management for Banks

A practical guide to third-party AI vendor risk assessment for NYDFS-regulated banks — covering LLM vulnerabilities, contract controls, due diligence frameworks, and ongoing monitoring requirements.

12 min read

Banks have always managed third-party risk. What has changed is the nature of what those third parties now deliver. When a vendor supplies a generative AI model — whether a customer-facing chatbot, a document summarization tool, or a credit analysis assistant — the risk profile looks nothing like a traditional software integration. A third-party AI vendor risk assessment must account for probabilistic outputs, opaque training pipelines, and attack surfaces that did not exist five years ago. For NYDFS-regulated institutions, the stakes are regulatory as well as operational: the Department’s evolving cybersecurity guidance makes clear that AI-related vendor risk falls squarely within the covered entity’s accountability perimeter. Understanding how those obligations are framed starts with a grounding in NYDFS AI Cybersecurity Guidance Compliance — the framework that shapes every control discussed below.

Why Third-Party AI Vendors Create Unique Risk Exposure for Banks

Standard vendor due diligence asks whether a supplier has SOC 2 Type II certification, adequate business continuity plans, and a reasonable incident response SLA. Those questions remain necessary but are no longer sufficient on their own. Generative AI vendors introduce risk categories that conventional IT vendor reviews were never designed to surface. The model itself is a dynamic artifact — it can behave differently on Tuesday than it did on Monday, not because anyone changed the code, but because the underlying system has been fine-tuned, the inference infrastructure has been updated, or the prompt context has shifted. That variability is not a bug in the traditional sense; it is intrinsic to how large language models operate. For a bank, variability in a core system is a compliance and safety problem. The generative AI security risks facing banking institutions also extend to supply chain opacity. When a bank integrates a third-party LLM, it is often inheriting risk from the model developer’s training data sourcing, the cloud provider running inference, and any plugins or retrieval-augmented generation (RAG) components the vendor has bolted on. Each of those layers introduces exposure that the bank’s security team cannot directly inspect or remediate. NYDFS has signaled — through its 2023 cybersecurity regulation amendments and subsequent guidance — that regulated entities are responsible for the cybersecurity posture of their third-party service providers. AI cybersecurity threats in financial services are not carved out from that obligation. Institutions that treat an AI vendor’s attestation letter as the end of due diligence are exposed. For a grounding overview of how NYDFS frames AI-related obligations, see NYDFS AI Cybersecurity Guidance: What Banks and Financial Institutions Must Know.

Mapping the Threat Surface: LLM-Specific Vulnerabilities Banks Must Evaluate

A thorough third-party LLM vendor assessment requires evaluating at least four categories of technical risk that have no direct analog in traditional software procurement.

  • Hallucination and output reliability. Large language models generate plausible-sounding text that can be factually wrong, legally incorrect, or internally inconsistent. In a banking context — loan underwriting summaries, regulatory filing drafts, customer-facing disclosures — a hallucinated output is not merely embarrassing. It can constitute a material misrepresentation. AI hallucination risk mitigation at the vendor level should include documented accuracy benchmarks on domain-specific tasks, red-team testing results, and a clear statement of which use cases the vendor does and does not support.
  • Training data leakage. If a vendor’s model was trained on data that included personally identifiable information, proprietary financial data, or confidential communications, that information may be recoverable through adversarial prompting. Banks should require vendors to document training data provenance, confirm that customer data submitted via the API is not used for model training without explicit consent, and provide contractual indemnification for data exposure events.
  • Prompt injection. This is among the most underappreciated machine learning security vulnerabilities in enterprise AI deployments. An attacker — or a malicious document processed by the system — can embed instructions in the input that override the model’s intended behavior. In a bank’s document processing workflow, a prompt injection attack could cause the AI to exfiltrate data, generate misleading outputs, or bypass controls. Vendors should be able to demonstrate input sanitization, output filtering, and logging that would surface injection attempts. The OWASP Top 10 for Large Language Model Applications catalogs these and related vulnerabilities and is a useful reference when structuring vendor questionnaires.
  • Model drift and version control. Unlike traditional software, LLM behavior can shift when the underlying model is updated — even when the API endpoint and version number appear unchanged. Banks need contractual rights to be notified before model updates that could affect output behavior, along with the ability to pin to a specific model version for regulated workflows.

These vulnerabilities connect directly to the broader model risk management obligations covered in AI Model Risk Management and Validation Requirements Under NYDFS.

Data Privacy and Confidentiality Controls in AI Vendor Contracts

Every time a bank employee pastes a customer record into a third-party AI tool, or an automated workflow sends transaction data to an external LLM API, that data leaves the institution’s direct control. AI training data privacy protection is not a theoretical concern — it is an active compliance obligation under GLBA, state privacy laws, and NYDFS cybersecurity requirements. Contracts with AI vendors should address the following at minimum:

  • Data processing restrictions. The agreement must explicitly prohibit the vendor from using customer data, employee data, or proprietary bank data to train or fine-tune models. This is a non-negotiable term for any institution subject to GLBA or NYDFS oversight. The ChatGPT compliance questions that surfaced across financial services in 2023 — when several institutions banned employee use of public LLM tools — were largely about this exact issue: the default terms of consumer AI products permitted training on user inputs.
  • Data residency and access controls. Specify where data is processed and stored, which vendor personnel can access it, and under what circumstances. For institutions with international operations or customers, cross-border data transfer restrictions may apply.
  • Encryption and tokenization. Require that data in transit and at rest is encrypted to current standards. For sensitive fields, evaluate whether tokenization or pseudonymization can reduce exposure before data reaches the vendor’s inference environment.
  • Audit rights. Banks should retain the right to audit vendor compliance with data handling terms, either directly or through a qualified third party. This is consistent with NYDFS expectations for third-party oversight and is increasingly standard in enterprise AI contracts.
  • Breach notification timelines. AI data security compliance in banking requires that vendors commit to notification windows that allow the institution to meet its own regulatory reporting obligations — typically 72 hours under NYDFS cybersecurity regulations.

Building a Third-Party AI Vendor Assessment Framework Under NYDFS

A structured third-party AI vendor risk assessment process moves through three phases: pre-onboarding due diligence, risk tiering and approval, and contractual controls. Treating these as sequential rather than parallel disciplines is where many institutions lose ground — by the time legal is reviewing the contract, risk tiering assumptions made weeks earlier may no longer reflect what the vendor actually does.

  • Pre-onboarding due diligence begins before any AI vendor is approved for use with bank data or customer-facing workflows. The assessment team should collect model documentation (architecture overview, training data description, known limitations, and benchmark results on relevant tasks), security certifications such as SOC 2 Type II or ISO 27001 with AI-specific controls noted, penetration testing results including adversarial prompt testing and injection resistance, a full subprocessor list covering all third parties the vendor relies on for model training or inference, and the vendor’s regulatory compliance posture across GDPR, CCPA, GLBA, and any sector-specific certifications.
  • Risk tiering follows once that documentation is in hand. Not all AI vendors carry equal risk — a vendor providing a grammar-checking tool for internal documents sits in a different tier than one providing credit decisioning support. Generative AI governance in banking requires a tiering methodology that accounts for the sensitivity of data processed, the degree of human oversight over AI outputs before they affect customers or decisions, the regulatory classification of the use case, and the vendor’s systemic importance to bank operations. Higher-tier vendors face more intensive due diligence, more frequent reassessment, and stronger contractual protections.
  • Contractual controls and approval close the loop. LLM risk management in banking requires that the vendor contract codify every control identified in due diligence. Legal and compliance teams should review AI vendor agreements with the same scrutiny applied to core banking system contracts — because for the use cases where these tools are deployed, the exposure is comparable.

The NYDFS AI Compliance Checklist and Implementation Roadmap for Banks provides a structured template for mapping these requirements against the Department’s specific expectations.

Ongoing Monitoring, Incident Response, and Vendor Exit Planning

Onboarding approval is not the end of the risk management lifecycle. Generative AI governance in banking requires continuous oversight because the technology — and the threat landscape around it — evolves faster than annual review cycles can track.

  • Continuous monitoring obligations under NYDFS extend to third-party service providers on an ongoing basis, not just at contract renewal. For AI vendors, this means tracking vendor security advisories and model update notifications, reviewing output quality on a sample basis to detect drift or degradation, monitoring for new vulnerabilities in the LLM ecosystem, and reassessing risk tier when the vendor’s use case expands or the vendor is acquired.
  • AI-specific incident triggers require extending standard incident response playbooks beyond data breaches and system outages. AI data security compliance in banking means those playbooks must also cover a prompt injection attack that caused the model to output confidential data, a hallucinated output that was acted on by a customer or employee, or a vendor disclosure that training data included improperly sourced information. Each of these should have a defined escalation path, a documentation requirement, and a regulatory notification assessment.
  • Vendor exit planning is often negotiated away in the rush to close a vendor agreement, but for AI vendors it is particularly important because data portability and model dependency create lock-in risks. Banks should require the right to terminate for cause without penalty if the vendor suffers a material security incident, data deletion certification upon contract termination, a transition period with continued access to support migration to an alternative, and a prohibition on the vendor retaining any bank data post-termination. LLM risk management in banking is not complete without a credible answer to the question: what happens if this vendor fails, is acquired, or becomes non-compliant?

For a broader view of the tools that support these ongoing obligations, see AI Governance Tools and Platforms for NYDFS-Regulated Banks.

Get the AI Vendor Due Diligence Checklist Managing third-party AI vendor risk under NYDFS requires more than a questionnaire. Download our gated AI vendor due diligence checklist — built for compliance and security teams at NYDFS-regulated institutions — or schedule a consultation to walk through your current vendor assessment process. Download the Checklist | Schedule a Consultation

Brine

Build, hire, and govern every AI system in your environment.

Describe a workflow in plain English and Brine builds it, runs it on your data, and governs every step — costing and attributing each action to its agent and model, with a pre-dispatch cap that holds a step before it overspends and a signed audit trail as standard.

Scope a pilot →More resources