Fine-tuning open models for financial analysis
Evaluation tells us which models to use. Training research tells us how far we can push the parts of the pipeline that matter most for grounded, domain-specific output.
Choosing the right model is one half of the work. The other half is understanding what these models can be taught. Beyond evaluating off-the-shelf models, we run our own training research on open base models, focused narrowly on the kind of financial work Perch is built for.
This is not about replacing a general model with a fine-tune. It is about learning, in a controlled setting, where domain adaptation helps and where it does not.
Parameter-efficient fine-tuning
Our first line of training research uses low-rank adaptation, a parameter-efficient method that trains a small set of additional weights on top of a frozen base model rather than retraining the whole network. It is cheap enough to iterate on quickly and small enough to keep and compare many variants.
We built a LoRA adapter on Llama 3.1 8B for equity and volatility analysis, trained on a curated, real-world market dataset. The adapter targets the attention and feed-forward projections across the model, with a modest rank, which is enough capacity to shift behavior on the target domain without overwriting the base model's general ability.
The goal was deliberately narrow: better grounding on domain documents, cleaner structured output, and steadier behavior on the specific shapes of data that financial work produces. Narrow goals are easier to measure and harder to fool than broad ones.
Domain-adaptive pretraining
The second line is domain-adaptive pretraining, where a base model continues training on a large, unlabeled corpus from the target domain before any task-specific tuning. Where low-rank adaptation teaches a model to behave a certain way on a task, domain-adaptive pretraining shifts the model's underlying familiarity with the domain's language and structure.
We ran domain-adaptive pretraining experiments on Qwen to study that shift directly: how much domain exposure changes grounding and terminology handling, and where the returns flatten out.
What we are actually measuring
Training a model is easy. Knowing whether it got better at the thing you care about is the hard part, and it is the same discipline we apply when evaluating any model. For this research that means asking:
- Does it ground claims in the source material more reliably, or just sound more fluent in the domain?
- Is the structured output cleaner and more consistent, or only different?
- Does it hold up on documents it was not trained on, or has it narrowed?
A fine-tune that sounds more like a financial analyst but grounds its claims no better is not progress. It is a more convincing way to be wrong, which is exactly the failure mode we are trying to design out.
Why this work informs the product
This research does not mean a custom model powers every answer. It means we understand, from the inside, what these models can and cannot be taught about a domain, and that understanding shapes how Perch routes and grounds work today. The parts of the pipeline that matter most for verifiable, decision-useful output are the parts we are willing to do the slow, empirical work on.
For the broader picture of why this matters, see why most AI agents fail in production.
Frequently asked questions
- What is the difference between LoRA fine-tuning and domain-adaptive pretraining?
- Low-rank adaptation trains a small set of additional weights on top of a frozen base model, which teaches it to behave a certain way on a specific task. Domain-adaptive pretraining continues training a base model on a large unlabeled corpus from the target domain before any task-specific tuning, which shifts the model's underlying familiarity with the domain's language and structure. The first changes how a model behaves on a task. The second changes what the model finds familiar in the first place.
- What is parameter-efficient fine-tuning?
- It is a family of methods that adapt a model by training a small number of additional parameters rather than retraining the whole network, with low-rank adaptation as the common example. The practical appeal is that it is cheap enough to iterate on quickly and small enough to keep and compare many variants. Our adapter targets the attention and feed-forward projections across the model at a modest rank, which is enough capacity to shift behavior on a target domain without overwriting the base model's general ability.
- Does fine-tuning a model on financial data make it more accurate?
- Not necessarily, and this is the trap the research is designed to catch. A fine-tune can make a model sound more like a financial analyst without grounding its claims in the source material any better than before. That is not progress, it is a more convincing way to be wrong, which is exactly the failure mode worth designing out. The questions that separate the two are whether it grounds claims more reliably rather than just sounding more fluent, whether structured output is genuinely cleaner rather than merely different, and whether it holds up on documents it was not trained on rather than having narrowed.
- Should you fine-tune an open model for financial analysis?
- It depends on what you want from it, and the honest answer is that domain adaptation helps in some places and not others. Our own goals are deliberately narrow, targeting better grounding on domain documents, cleaner structured output, and steadier behavior on the specific shapes of data that financial work produces. Narrow goals are easier to measure and harder to fool than broad ones, so a project that cannot say precisely what it expects to improve will struggle to tell whether it did.
- Does Perch run on a custom fine-tuned model?
- No. This research does not mean a custom model powers every answer. It means we understand, from the inside, what these models can and cannot be taught about a domain, and that understanding shapes how Perch routes and grounds work today. We built a low-rank adaptation adapter on Llama 3.1 8B for equity and volatility analysis and ran domain-adaptive pretraining experiments on Qwen, as controlled studies rather than as production models.