Parameter-efficient fine-tuning (PEFT) methods like LoRA and QLoRA have democratized AI fine-tuning for financial institutions. Where full fine-tuning of a 70B parameter model once required $100,000+ in GPU compute, LoRA can achieve comparable results at 5-10% of the cost. This guide explains how these techniques work and how to apply them to financial AI use cases.
Why PEFT Methods Matter for Finance
Full fine-tuning of large language models requires enormous GPU resources. Fine-tuning a 70B parameter model from scratch costs $10,000 to $100,000 per training run, making it inaccessible to all but the largest financial institutions. PEFT methods like LoRA reduce this cost by 10-20x, bringing fine-tuning within reach of mid-sized banks, asset managers, and fintech companies.
Beyond cost, PEFT methods offer practical advantages for financial institutions. The resulting adapter files are small (megabytes rather than gigabytes), making them easy to version control, deploy, and audit. Multiple adapters can be swapped at inference time, allowing a single base model to serve multiple financial use cases. And because the base model remains unchanged, PEFT methods preserve the model's general capabilities while adding financial specialization.
How LoRA Works
LoRA (Low-Rank Adaptation) is based on a simple insight: the weight updates needed to adapt a model to a new domain have a low intrinsic rank. Instead of updating all of a model's weights during fine-tuning, LoRA adds small trainable matrices (adapters) to specific layers of the frozen base model. These adapters have far fewer parameters than the full model, making training fast and memory-efficient.
For financial text, the rank parameter (r) controls how much adaptation capacity the model has. A rank of 8-16 is typically sufficient for financial domain adaptation. Higher ranks (32-64) may be needed for complex financial tasks with diverse output formats. The alpha parameter controls the scaling of the adapter updates and should be set to 2x the rank value as a starting point.
Which layers to target for financial text depends on the model architecture. For Llama-based models, targeting the query, key, value, and output projection matrices in the attention layers provides the best results. Adding adapter modules to the feed-forward layers can further improve performance on financial classification tasks.
How QLoRA Works
QLoRA combines 4-bit NormalFloat quantization with LoRA to enable fine-tuning on consumer-grade GPUs. The base model is quantized to 4-bit precision, reducing memory requirements by 4x, while the LoRA adapters remain in full precision. This allows fine-tuning of a 70B parameter model on a single 24GB GPU — a task that would normally require 140GB of GPU memory.
The trade-off with QLoRA is a slight reduction in model quality. The quantization process introduces some information loss, and the 4-bit representation limits the model's ability to represent fine-grained financial distinctions. In practice, the quality loss is small (1-3% on most benchmarks) and is acceptable for many financial use cases, especially proof-of-concept projects and non-customer-facing applications.
Memory requirements for QLoRA vary by model size. A 7B parameter model requires approximately 6-8GB of GPU memory, fitting on most consumer GPUs. A 70B model requires 20-24GB, fitting on high-end consumer GPUs like the RTX 4090. Training time is 2-3x longer than LoRA due to the quantization/dequantization overhead.
LoRA vs QLoRA for Financial Institutions
LoRA requires more GPU memory but delivers higher quality. A LoRA fine-tune on a 70B model requires 4 H100 GPUs and completes in 4-8 hours. Quality is typically within 1% of full fine-tuning on financial benchmarks. LoRA is recommended for production financial systems where quality is critical.
QLoRA enables fine-tuning on limited hardware but with quality trade-offs. A QLoRA fine-tune on a 70B model can run on a single 24GB GPU but takes 24-48 hours. Quality is 1-3% below full fine-tuning. QLoRA is recommended for proofs-of-concept, smaller institutions, and non-customer-facing applications.
For regulated financial institutions, LoRA is generally preferred over QLoRA because it maintains full model precision. The additional GPU investment is justified by the quality improvement and reduced validation burden. However, QLoRA is an excellent option for smaller institutions and teams that need to demonstrate value before investing in enterprise GPU infrastructure.
Practical LoRA Configuration for Finance
Recommended rank (r) values by use case: for financial document classification, r=8 is sufficient. For credit memo generation, r=16 provides better output quality. For complex financial analysis tasks, r=32 may be needed. Start with r=16 and increase if the model's outputs lack the specificity you need.
Alpha should typically be set to 2x the rank value. For r=16, use alpha=32. The dropout rate should be 0.05-0.1 for financial text. Higher dropout can help prevent overfitting when training data is limited, but may reduce the model's ability to learn complex financial patterns.
Target modules for financial LLMs: for Llama models, target "q_proj", "k_proj", "v_proj", "o_proj" in attention layers. For Mistral models, the same module names apply. For Phi models, target "q_proj", "k_proj", "v_proj", "dense". Adding feed-forward modules like "gate_proj", "up_proj", "down_proj" can improve classification performance.
Training duration: 2-4 epochs is typically sufficient for financial fine-tuning. More epochs risk overfitting, especially with smaller datasets. Use early stopping based on validation loss to find the optimal training duration. Learning rate should be 1e-4 to 5e-4 for LoRA, with a cosine scheduler that warms up over the first 10% of steps.
Deployment of LoRA Adapters
Merging adapters into the base model is the simplest deployment approach. The adapter weights are merged into the model's weights, creating a single model file that can be served like any other model. This approach is recommended for production deployment where inference latency is critical.
Serving with Ollama or NVIDIA NIM provides a production-grade inference API. Ollama supports LoRA adapters natively and can swap between adapters at runtime. NVIDIA NIM provides optimized inference with support for LoRA adapters in enterprise deployments.
Version control for regulatory compliance requires tracking both the base model and adapter versions. Maintain a model registry that records which base model, which adapter, and which training data were used for each deployed model version. This documentation is essential for SR 11-7 compliance.