Should I fine-tune a model or use context files for my codebase?

For most teams, context files. Fine-tuning bakes knowledge into the model's weights — expensive, slow to update, and tied to a model you don't fully control. Context files (a contextbase the agent reads before acting) are cheap, instantly updatable, version-controlled, and work with any model. Context wins unless you have a very specific reason to fine-tune.

What's the difference between fine-tuning and context engineering?

Fine-tuning changes the model itself by training it on your data, so the knowledge is baked into the weights. Context engineering leaves the model alone and supplies knowledge at runtime, in files the model reads before acting. One edits the brain; the other hands it a briefing. For evolving codebase knowledge, the briefing is far more practical.

When is fine-tuning worth it?

When you need the model to internalise a stable style, format, or domain that rarely changes and is hard to express as context — and you have the data and budget to do it well. For knowledge that changes constantly, like a living codebase, fine-tuning goes stale immediately, which is exactly where context files shine.

ArticlesComparison

Context files vs fine-tuning for your codebase

Want an agent that knows your codebase? Fine-tune a model or write context files. Why context wins for most teams.

Stuart LeoJune 9, 20265 min read

You want an agent that knows your codebase — your patterns, your decisions, your conventions — instead of one that guesses like a stranger. There are two ways to get there, and they're often confused: fine-tune a model on your code, or write context files the model reads. They sound similar. They're very different bets, and for most teams one of them is clearly right.

Here's the honest comparison of fine-tuning versus context, and why context wins for almost everyone building software.

Two ways to teach an agent your code

The goal is the same: an agent that behaves as if it already knows your project. The two routes get there differently.

Fine-tuning changes the model itself. You train it further on your codebase and conventions, so the knowledge is baked into the weights. You've edited the brain.
Context files leave the model untouched. You write your knowledge into files the agent reads before it acts — a contextbase. You've handed the brain a briefing.

One alters the model. The other informs it at runtime. That single difference drives everything else.

Fine-tuning: bake it into the model

Fine-tuning's appeal is that the knowledge becomes part of the model — no need to supply it each time. For a stable, hard-to-describe style or domain, that can be genuinely valuable.

But the costs are steep for codebase knowledge specifically. It needs data, expertise, and budget. It produces a model artifact you have to host and manage. And — the killer — it's slow to update. Your codebase changes daily — the fine-tuned model is frozen at its last training run. The moment you make a decision the model wasn't trained on, it's out of date, and updating means another training cycle. You've baked a snapshot of a moving target.

Context files: read it before acting

Context files take the opposite approach: keep the knowledge outside the model, in version-controlled markdown the agent reads before each task. The discipline of curating that knowledge well is context engineering, and it has a decisive practical edge for anything that changes.

Update a context file and the change is live on the very next task — no training, no cost, no waiting. The knowledge is plain text you can read, edit, review, and version like code. And it's yours, sitting in your repo, not encoded in a model's weights you don't control.

Stuart Leo

Fine-tuning bakes a snapshot of your codebase into a model. Context files keep a living copy of it in your repo — current, readable, and yours.

Cost, speed, and staleness

Lined up, the comparison is lopsided for evolving codebase knowledge:

	Fine-tuning	Context files
Where knowledge lives	The model's weights	Files in your repo
Cost to set up	High (data, compute, expertise)	Low (write markdown)
Time to update	A training cycle	Instant — next task
Staleness	Frozen at last training	Always current
Portability	Tied to that model	Works with any model
Inspectable	No	Yes — it's plain text
You own it	The provider's model, your tuning	Entirely yours

Why context wins for most teams

For a living codebase — which is every codebase — the staleness problem alone settles it. Knowledge that changes daily can't be baked into a model that updates monthly. Context files track the change for free, the day it happens.

Add that they're cheaper, readable, reviewable, model-agnostic (so you can route or switch models freely), and genuinely yours, and the case is clear: for teaching an agent your codebase, write context, don't fine-tune. (Fine-tuning still earns its place for stable style or domain internalisation — just not for the moving target of your code.)

Fine-tuning bakes knowledge into a model you don't own — context files keep it yours, current, and free to change.

Start here: see what a contextbase is, context engineering vs RAG, or read the method.

FAQ

Should I fine-tune a model or use context files for my codebase?: For most teams, context files. Fine-tuning bakes knowledge into the model's weights — expensive, slow to update, and tied to a model you don't fully control. Context files (a contextbase the agent reads before acting) are cheap, instantly updatable, version-controlled, and work with any model. Context wins unless you have a very specific reason to fine-tune.
What's the difference between fine-tuning and context engineering?: Fine-tuning changes the model itself by training it on your data, so the knowledge is baked into the weights. Context engineering leaves the model alone and supplies knowledge at runtime, in files the model reads before acting. One edits the brain; the other hands it a briefing. For evolving codebase knowledge, the briefing is far more practical.
When is fine-tuning worth it?: When you need the model to internalise a stable style, format, or domain that rarely changes and is hard to express as context — and you have the data and budget to do it well. For knowledge that changes constantly, like a living codebase, fine-tuning goes stale immediately, which is exactly where context files shine.

What is a contextbase? The asset that makes agents remember

A contextbase is the version-controlled folder of markdown your AI coding agent reads before it acts. What goes in it, why it compounds, and how to start one today.

Context engineering vs RAG for coding agents

RAG retrieves context on the fly; context engineering curates and keeps it. How they differ for coding agents, where each fits, and why a contextbase isn't RAG.

Context engineering, explained: the skill that replaced prompting

Context engineering is the discipline that overtook prompt engineering: curating what your AI agent knows before it acts. What it is, why it matters, and how to start.

All articles