LLM Distillation for Financial Reports

Problem

We used distillation to develop a financial-analysis tool that is computationally efficient and simpler than traditional LLMs, while being specialized for financial contexts. LLMs cannot be efficiently used and hosted on an ad-hoc basis — so we aimed to train smaller models that are easily accessible.

Why It Matters

Financial-analysis tools are essential for evaluating a company's fiscal health. Current tools demand significant computational resources while remaining too general to stay consistently accurate in financial contexts. Distillation combines the strengths of multiple models while being far more resource-efficient — making advanced financial analysis accessible, and improving decision-making across industries.

Methodology — The Pipeline

Take a large unified financial dataset, preprocess it, and hand it to big, renowned LLMs — Meta's LLaMA 3 and Claude 3.5 Haiku. The teachers don't just answer: they produce a label and a rationale. The student — a much smaller model like Flan-T5 or GPT-2 — trains on both. It learns the reasoning, not just the answers.

Watch It Think

Real outputs from our runs, verbatim — the standing prompt is “What is the sentiment of this news? Please choose an answer from {negative/neutral/positive}.” Note the student doesn't just classify; it explains itself, the way its teachers taught it to.

Quirks included — “disaastah” is the actual test sentence, and the student's explanation is untouched.

Discussion

LLaMA 3 and Claude 3.5 excel in similar areas, making distillation effective for combining their strengths. T5 distilled from LLaMA 3 outperforms FinGPT — indicating that step-by-step distillation is more effective than fine-tuning or LoRAs, offering cost-efficient performance and suggesting distillation is a promising optimization strategy.

Results

We provided a model that outperforms current LoRAs and fine-tuned GPTs using less than 12% of the original dataset (Sentiment Train FinGPT) on the FPB benchmark. It not only takes less training time, but inference is dramatically faster — making the model usable in resource-constrained systems.

Distilled, and still winning

On the Financial Phrase Bank benchmark, the distilled T5 doesn't just approach its teachers — it edges past them, at a fraction of the size and trained on under 12% of the data. Toggle the metric: