On-Premise AI for Financial Services: A Compliance-First Architecture Guide

The Regulatory Landscape Financial Institutions Are Navigating

Before discussing architecture, it is worth being specific about which regulations shape AI deployment decisions in European financial services.

DORA (Digital Operational Resilience Act) applies from January 2025 and imposes obligations on ICT risk management, third-party provider oversight, incident reporting, and operational resilience testing. For AI deployments, the most relevant provisions concern concentration risk — supervisors are specifically concerned about systemic dependency on a small number of large cloud providers — and the requirement for contractual rights to audit and test third-party ICT systems. Running AI on infrastructure managed by a major hyperscaler creates DORA obligations that must be explicitly managed.

MiFID II and MiFID III affect how AI can be used in investment services, including algorithmic trading, client suitability assessments, and communications monitoring. The explainability requirements for automated decisions and the record-keeping obligations for client interactions create specific audit trail requirements for AI systems.

GDPR governs how personal data is processed by AI systems, including the right to explanation for automated decisions that affect individuals, restrictions on cross-border data transfers, and data minimisation principles. Every AI interaction involving customer data is a data processing activity requiring a lawful basis and appropriate safeguards.

EU AI Act classifies certain financial services AI applications as high-risk — including credit scoring, creditworthiness assessment, and AI used to evaluate eligibility for essential services. High-risk AI systems face obligations around documentation, human oversight, accuracy, robustness, and cybersecurity.

On-premise deployment does not automatically satisfy these requirements, but it creates the conditions under which they can be practically met. A financial institution cannot produce audit evidence about a cloud AI system it does not fully control.

Why Cloud AI Creates Specific Risks for Financial Institutions

Cloud AI services — including large language model APIs and cloud-hosted agent platforms — introduce several risk categories that are particularly difficult to manage in a financial services context.

Data residency uncertainty. Sending customer financial data, trading information, or internal documents to a cloud AI API means that data is processed on infrastructure you do not control, in jurisdictions that may change with vendor decisions. For firms with strict data residency requirements — common under GDPR, Swiss data protection law, and national implementations — this creates continuous compliance exposure that requires ongoing monitoring rather than one-time assessment.

Third-party concentration risk. DORA explicitly requires financial institutions to assess and manage ICT concentration risk. A firm that processes significant AI workloads through a single hyperscaler or a single large language model provider has a concentration risk that supervisors will examine. On-premise AI distributes this risk back to institutional infrastructure.

Audit access limitations. Regulatory examinations in financial services frequently require firms to produce evidence about system behaviour — what the system did, when, with what data, and with what outcome. Cloud AI providers may not offer audit log export in the format, granularity, or retention period required. Some providers explicitly limit audit capabilities in their terms of service.

Model governance opacity. Cloud AI models change. Providers update, retrain, and deprecate models on schedules and with behaviours that cloud customers cannot always predict or control. For financial services firms operating under model risk management frameworks (such as SR 11-7 guidance or EBA guidelines on internal model governance), uncontrolled model changes create validation and documentation obligations that are difficult to meet when the model is managed externally.

Vendor lock-in and exit risk. DORA requires exit plans for critical third-party providers. For AI workloads where institutional processes have become dependent on a specific cloud model or API, exit planning is complex. On-premise deployments with open-weight models are structurally easier to migrate.

What On-Premise AI Architecture Looks Like for Financial Services

An on-premise AI platform for a regulated financial institution is not a single product but a layered architecture. The core components are:

Private model inference. Open-weight large language models (such as LLaMA 3, Mistral, or domain-specific financial models) running on GPU infrastructure inside the institutional data centre or private cloud. No customer data or internal documents leave the institutional perimeter. Model versions are controlled, documented, and validated before deployment.

Private RAG (Retrieval-Augmented Generation). A document retrieval layer that allows AI agents to access internal knowledge bases — policy documents, regulatory guidance, product documentation, client agreements — without sending document content to external systems. The vector index and retrieval infrastructure are managed within the institution.

Agent orchestration with governance controls. An orchestration layer that routes tasks to appropriate AI agents, enforces access controls based on user roles, logs every interaction with full provenance, and supports human oversight workflows including approval gates for high-risk outputs. This is the layer where EU AI Act and MiFID II obligations about human control are operationalised.

Audit logging and explainability. Structured, exportable logs covering every AI interaction: user identity, input data, model and version, retrieved documents, output, timestamp, and any human review actions. These logs are the evidence package that compliance officers and regulators require.

Model governance tooling. Version control for deployed models, documentation of training data and known limitations, validation records, and change management workflows. This supports model risk management frameworks and EU AI Act documentation obligations for high-risk AI systems.

High-Value Use Cases in Financial Services

On-premise AI in financial services is not limited to a single application. The platform serves multiple use cases simultaneously while maintaining consistent governance:

Regulatory compliance Q&A. Staff can query internal policy libraries, regulatory guidance, and compliance documentation in natural language. The system retrieves relevant passages and synthesises answers without sending sensitive internal documents externally. Compliance officers can get answers to complex regulatory questions faster than manual search allows, with full audit trails of what was asked and what was retrieved.

AML and fraud explanation. AI agents can explain the reasoning behind AML alerts and fraud flags to investigators — providing the context, transaction patterns, and policy references that support human decision-making. This supports the explainability requirements in EU AML regulation and the human oversight provisions in the EU AI Act.

Client onboarding document processing. AI agents can extract, classify, and verify information from onboarding documents — KYC forms, identity documents, beneficial ownership declarations — within a secure perimeter. Customer data does not leave the institution during processing.

Trade reporting and reconciliation assistance. AI agents can assist with the complex, rules-heavy process of regulatory trade reporting, helping operations teams identify errors, understand reporting obligations, and resolve reconciliation issues.

Internal knowledge management. Large financial institutions contain enormous volumes of internal knowledge — legal opinions, product guidelines, process documentation, regulatory interpretations — that is difficult to access and apply consistently. Private RAG makes this knowledge available at the point of need without creating new compliance risks.

Risk model documentation. AI agents can assist with the documentation requirements for model risk management — generating initial drafts of model use documentation, identifying gaps in validation evidence, and maintaining consistent documentation standards across the model inventory.

Meeting EU AI Act Obligations with On-Premise Architecture

The EU AI Act classifies several financial services AI applications as high-risk, including systems used for creditworthiness assessment, credit scoring, and evaluating eligibility for financial products and services. High-risk AI systems face specific obligations:

• Risk management system throughout the AI system lifecycle
• Data governance covering training, validation, and testing datasets
• Technical documentation sufficient to allow regulatory assessment
• Logging and traceability with automatic recording of events throughout the system lifecycle
• Transparency and information provision to deployers
• Human oversight measures that allow qualified persons to monitor and intervene
• Accuracy, robustness, and cybersecurity requirements

On-premise deployment facilitates each of these obligations by keeping the AI system and its operational data within institutional control. The institution can maintain and produce documentation, configure logging to the required granularity, implement human oversight workflows within its existing governance structures, and conduct cybersecurity assessments on infrastructure it directly manages.

For AI systems that are not high-risk but still process sensitive financial data, on-premise deployment remains the lower-risk architecture from a data protection and third-party risk management perspective.

Implementation Considerations

Deploying on-premise AI in a regulated financial institution requires planning across several dimensions:

Infrastructure. GPU compute for model inference, high-performance storage for vector indexes and document repositories, and reliable networking. The infrastructure requirements depend on the number of concurrent users, the volume of documents in the knowledge base, and the latency requirements of the use cases.

Model selection and validation. Choosing appropriate open-weight models for financial services use cases, validating them against institutional model risk management frameworks, and documenting their limitations before deployment. Domain-specific fine-tuned models may outperform general models for specific financial tasks.

Integration with existing systems. Connecting the AI platform to document management systems, compliance databases, CRM, and core banking or trading systems through secure APIs with appropriate access controls and logging.

Staff training and change management. Ensuring that staff understand how to use AI tools appropriately, recognise the limitations of AI outputs, and know when to escalate to human experts. This is also an EU AI Act obligation for deployers of high-risk AI systems.

Governance and oversight. Establishing clear ownership of the AI platform, defining the governance processes for model deployment and change, and integrating AI system oversight into existing risk management frameworks.

VDF AI’s on-premise platform is designed for this environment. It runs entirely within your infrastructure, produces the audit logs and documentation that regulators require, and supports the human oversight workflows that distinguish compliant AI deployment from the alternatives.

Conclusion

Financial services firms are not anti-AI. They are pro-compliance, and those priorities used to conflict. On-premise AI architecture resolves much of that conflict by keeping data under institutional control, enabling the audit and documentation that regulators require, and supporting the governance structures that responsible AI deployment demands.

The institutions that will lead on AI in financial services are not those that moved fastest to cloud AI and later had to remediate compliance gaps. They are those that built on-premise foundations that allow AI to scale without accumulating regulatory risk with every new use case.