Your agent doesn’t need a better prompt.

I think almost everyone building AI agents goes through the same phase.

You build something simple. It works.

You ask a few questions you had in mind while building it, and it answers beautifully.

You think you’re almost done.

Then someone asks a question you didn’t think about.

The agent goes off the rails.

So you do what everyone does. You open the system prompt.

You add another instruction. Maybe another.

Then a few examples. Then a couple more, because the first ones weren’t enough.

Now the prompt is getting pretty long — but the agent seems better.

Until someone asks another question.

Back to the prompt. Another rule. Another few-shot. Another paragraph explaining what the model should and shouldn’t do.

You are a helpful support agent for Acme.
Answer briefly and accurately.

Tools available:
  search_orders · refund · kb_lookup
Be polite.

After a while, the prompt becomes this giant document that you’re honestly afraid to touch. Every new sentence feels like you’re fixing one thing while secretly breaking three others.

The frustrating part is that none of the changes feel wrong. Each one makes sense on its own. But somehow the whole thing keeps getting less reliable.

If this sounds familiar, I don’t think your problem is the prompt. I think you’re asking the prompt to do a job it was never meant to do.

A thin agent in a thick harness.

Of everything I’ve learned shipping agents, one idea has changed the way I build them more than any prompting technique:

Build a thin agent sitting inside a thick harness.

The harness is everything around the model. The tools. The execution engine. The validators. The caches. The retry logic. The state management. The skill loader. The thing that quietly keeps the model on the road.

The model is still making decisions. But it isn’t responsible for remembering hundreds of rules, or carrying every piece of knowledge from the beginning of the conversation. That’s the harness’s job.

The more work your harness does, the less your prompt has to do.

And in my experience, that’s exactly what makes agents become more reliable.

Only what the next decision needs.

The first principle that follows from this is surprisingly simple.

Only give the model what it needs to make the next meaningful decision.

Not every rule.

Not every schema.

Not every recovery strategy.

Not every possible edge case.

Just enough information for the decision sitting in front of it.

Think about how people work. If I ask you to write a SQL query, I don’t first hand you a 400-page database manual. I tell you what table you’re working with. Maybe the schema. Maybe one example. That’s enough to get started.

orders(id, customer_id, total, created_at)
customers(id, name, region)

-- one example
SELECT … FROM orders
WHERE created_at > now() - interval '7 days'

If you hit an error, then we look at the error. If you need a special function, then we open that part of the documentation. We don’t study the whole database before writing line one.

Models aren’t very different.

When an agent starts, it doesn’t need to know how to recover from every OpenSearch error. It doesn’t need to know every Cypher pattern. It doesn’t need every tool description you’ve ever written. It needs to answer one question:

“What’s the next thing I should do?”

Maybe that next thing is deciding whether to search or plan. Maybe it’s choosing between a graph database and OpenSearch. Maybe it’s deciding that this task should first be decomposed.

Whatever that decision is — that’s the only context that deserves to be in front of the model.

Everything else can wait.