← Fintech GlossaryAI & LLM

Base Model (Foundation Model)

A base model, also known as a foundation model, is a large AI model pre-trained on vast and diverse datasets that serves as the starting point for fine-tuning or adaptation to specific tasks. Base models are trained on broad data at massive scale using self-supervised learning, giving them general language understanding, reasoning, and generation capabilities. Examples include Llama from Meta, GPT-4o from OpenAI, Claude from Anthropic, Gemini from Google, Mistral from Mistral AI, and DeepSeek. These models contain billions of parameters and are trained on trillions of tokens of text data. The defining characteristic of a base model is that it is not specialized for any particular task — it has general capabilities that can be directed toward specific applications through fine-tuning, prompt engineering, or RAG. The choice of base model is the most consequential decision in any AI fine-tuning project, as the base model's capabilities, limitations, and characteristics directly determine the ceiling of what the fine-tuned model can achieve. Key considerations when selecting a base model include its performance on relevant benchmarks, the breadth and quality of its pre-training data, its support for the languages and domains relevant to the target application, its inference speed and cost, and the availability of fine-tuning infrastructure and tooling. Base models are typically released under different licensing terms — open-source models like Llama and Mistral allow full customization including on-premise deployment, while proprietary models like GPT-4o and Claude can only be fine-tuned through the provider's API with restrictions on deployment and use.

In Financial Services

Selecting the right base model is the most important decision in any financial AI fine-tuning project. The base model determines the ceiling of achievable performance, the cost and complexity of fine-tuning, the deployment options available, and the regulatory compliance path. Financial institutions should evaluate base models on criteria specific to their requirements: financial knowledge capability, measured through performance on financial benchmarks; language support for the institution's operating languages; data residency options for regulatory compliance; licensing terms that permit the intended use case; and the availability of fine-tuning tooling and infrastructure. Open-source base models like Llama and Mistral offer financial institutions the most flexibility. They can be fine-tuned using any technique including LoRA, QLoRA, and full fine-tuning. They can be deployed on-premise for data residency compliance. The fine-tuned model can be fully owned and controlled by the institution. The trade-off is that open-source models typically lag behind frontier proprietary models on general capabilities, and the institution must manage the fine-tuning and deployment infrastructure. Proprietary base models like GPT-4o and Claude offer higher general capabilities but with restrictions. Fine-tuning is only available through the provider's API, limiting customization flexibility. The fine-tuned model must be deployed on the provider's infrastructure, which may not meet data residency requirements. The institution does not own the fine-tuned model weights. For financial applications requiring the highest possible accuracy on complex tasks — such as regulatory interpretation or sophisticated financial analysis — proprietary models may be preferred despite these limitations. The decision framework for financial institutions should consider the task complexity, accuracy requirements, data sensitivity, deployment environment, and budget. For most financial fine-tuning projects, an open-source model like Llama 3.3 70B or Mistral Large provides the best balance of capability, flexibility, and cost.

Real-World Example

A global asset manager with $500 billion in AUM evaluates four base models for fine-tuning a research analysis tool: Llama 3.3 70B, GPT-4o, Claude 3.5 Sonnet, and Mistral Large. The evaluation criteria include financial benchmark performance (weighted 30%), fine-tuning flexibility (25%), deployment cost (20%), data residency support (15%), and licensing terms (10%). Llama 3.3 70B scores highest on fine-tuning flexibility (supports LoRA, QLoRA, full fine-tuning, on-premise deployment, full model ownership) and deployment cost (zero inference API cost for on-premise). GPT-4o scores highest on financial benchmark performance (particularly complex reasoning tasks) but lowest on data residency and model ownership. The asset manager selects Llama 3.3 70B for its primary research analysis tool, fine-tuning using LoRA on 5,000 proprietary research examples. For a secondary, more complex natural language query application requiring interpretation of complex regulatory text, the manager also fine-tunes GPT-4o through OpenAI's API for 2,000 examples, accepting the data residency limitations for this specific use case. The dual-model approach gives the manager cost-effective performance for standard research tasks and highest accuracy for complex regulatory queries.

Why It Matters for Finance

Base model selection is the most consequential decision in financial AI fine-tuning because it determines the entire trajectory of the AI initiative. A poor base model choice cannot be overcome through fine-tuning — if the base model lacks financial knowledge, has poor language support, or cannot be deployed in the required jurisdiction, no amount of fine-tuning will fix these fundamental limitations. The decision framework for base model selection must balance multiple, often conflicting requirements. Open-source models offer flexibility, ownership, and data residency but may lag on capability. Proprietary models offer higher capability but restrict customization and deployment. The right choice depends on the specific use case, regulatory environment, and institutional capabilities. For most financial institutions, the recommended approach is to select a leading open-source model as the primary base model and use proprietary model APIs for specific tasks where highest accuracy is critical. This hybrid approach maximizes flexibility while accessing the best capabilities for the most demanding applications. The financial industry is increasingly converging on this approach, with Llama and Mistral as the most common open-source base models for fine-tuning and GPT-4o and Claude as the most common proprietary models for API-based use cases.

Explore in Finatune

Llama Claude Fine-Tuning Llama for Banking

Frequently Asked Questions

What is a base model in AI?

A base model (or foundation model) is a large AI model pre-trained on diverse data that serves as the starting point for fine-tuning. Examples include Llama, GPT-4o, Claude, and Mistral. Base models have general language understanding and reasoning capabilities that can be specialized through fine-tuning for specific financial tasks.

Which base models are best for financial AI fine-tuning?

Llama 3.3 70B is the most popular choice for financial fine-tuning due to its strong benchmark performance, open-source license, and broad tooling support. Mistral Large offers excellent multilingual capabilities for European markets. GPT-4o provides the highest general performance but only through API fine-tuning. The best choice depends on your accuracy needs, deployment flexibility, and data residency requirements.

How do I choose between open-source and commercial base models for finance?

Choose open-source models (Llama, Mistral) when you need on-premise deployment for data residency, full model ownership for regulatory compliance, maximum fine-tuning flexibility, or lower inference costs at scale. Choose commercial models (GPT-4o, Claude) when you need the highest accuracy on complex tasks, lack GPU infrastructure for self-hosting, or want to avoid managing model infrastructure. Many institutions use both.

← Previous Term: Attention Mechanism

Next Term: Catastrophic Forgetting →

View All Fintech Terms →