Where in the prompt things should go

The instruction is definitely in the prompt. You can see it. The model ignored it anyway.

Before rewriting the wording — which is where most people go next — check where it sits. Position within a long prompt affects how reliably content gets used, and the fix is often structural rather than rhetorical.

The phenomenon, described honestly

Models attend unevenly across a long context. Material at the beginning and end of the input is generally used more reliably than material buried in the middle. This is widely documented and commonly called the lost-in-the-middle effect.

Three honest caveats, because this gets overstated:

The magnitude varies by model and task. It is not a fixed penalty. Some models show it strongly, others much less.

Newer models generally handle it better. This is an active area of work and the situation has improved. Advice written two years ago overstates it for current models.

It depends on how much you’re asking of the middle. Retrieving one clearly-stated fact from a middle position is a different task from synthesising across several buried passages, and they degrade differently.

You will not find a specific percentage here, because any number would depend on the model, the task, the prompt length, and the year. What holds is the direction: the ends are more reliable than the middle, so put things that must be used at the ends.

Treat this as a prior to check against your own evaluation, not as a law.

A default layout

Reasonable starting point for a retrieval-augmented prompt:

┌─ System prompt ──────────────────────────┐
│ Role, capabilities, hard constraints     │   stable → cacheable
├─ Tool definitions ───────────────────────┤   stable → cacheable
├─ Retrieved passages ─────────────────────┤   varies per request
│   most relevant first, or first-and-last │
├─ Conversation history ───────────────────┤   grows
├─ Task instructions ──────────────────────┤   restated here
└─ Current user query ─────────────────────┘   last

The reasoning, section by section.

System prompt first. It’s stable across requests, which matters for caching (below). It also sets framing before anything else is read.

Tool definitions early. Stable, and part of the cacheable prefix.

Retrieved passages in the middle. Not because the middle is good, but because they’re bulky and something has to be there. Ordering within them matters, and that’s the next section.

History after passages. A judgement call. Putting history closer to the query means recent turns sit near the end, where they’re used most reliably — which is usually what you want, since recency correlates with relevance. Putting passages closer to the query instead is defensible when retrieval matters more than conversational continuity; test both if it’s load-bearing for you.

Task instructions restated near the end. Covered below.

The user’s actual query last. The final position is the most reliably attended, and the query is what everything else exists to serve.

Ordering within the retrieved set

If the middle is weaker, where do the best passages go?

Most relevant first is the common default and a fine choice. Simple, matches retriever output, easy to reason about.

First-and-last placement is the alternative worth knowing: put the highest-scoring passage first and the second-highest last, filling the middle with the rest. The idea is to occupy both strong positions with your best material rather than clustering everything strong at one end.

Which wins is task-dependent, and it’s cheap to test — it’s a reordering, not a redesign. Run both against your evaluation set.

Where relevance ordering does badly: tasks requiring synthesis across all passages. If the model must combine several sources, an ordering that emphasises one can bias the answer toward it. For synthesis tasks, ordering by something structural — chronology, document, source — is often better than by score, because it helps the model organise rather than rank.

Always label passages. Each one should carry a delimiter and an identifier — source, title, date. Two reasons: the model can cite and distinguish sources, and it can tell where one passage ends and the next begins. Unlabelled concatenated passages blur into one document and encourage the model to treat contradictory sources as a single confused one. This costs a handful of tokens per passage and is close to always worth it.

Restating instructions at the end

A specific and widely used technique: put the key instructions in the system prompt and again immediately before the user’s query.

[system prompt: role, constraints, format]
[tools]
[retrieved passages]
[history]

Answer using only the passages above. If they don't contain the
answer, say so. Cite the source ID for each claim.

[user query]

Why it works: by the time the model has read several thousand tokens of passages, the instructions from the top are far away and competing with everything since. Restating puts them in the strongest position, adjacent to the query they govern.

The cost is small — a few dozen tokens — and this is one of the highest-return changes available on a retrieval prompt.

Two cautions:

Keep the restatement short. It’s a reminder of the critical constraints, not a copy of the system prompt. Three to five lines.

Don’t contradict yourself. If the top says one thing and the bottom says another, behaviour becomes unpredictable. When you edit one, edit both — which is an argument for generating both from a single source in code rather than maintaining two strings.

What prompt caching constrains

If your provider supports prompt caching, it changes the layout calculus and adds a hard rule.

Caching typically works on a prefix basis: an identical leading portion of the prompt can be reused across requests at reduced cost and latency. The moment the content diverges from a previous request, caching stops for everything after that point.

The rule that follows: order by volatility, most stable first.

system prompt        ← never changes        ┐
tool definitions     ← changes on deploy    ├ cacheable prefix
static context       ← changes rarely       ┘
─────────────────────────────────────────────
retrieved passages   ← changes every request
history              ← grows every turn
instructions + query ← changes every request

Anything volatile placed early destroys the cacheability of everything after it. A timestamp at the top of the system prompt — easy to add without thinking — can invalidate the entire cache on every request.

Practical consequences:

Never put per-request data in the system prompt. No timestamps, no user IDs, no session identifiers, unless you genuinely need them there and have accepted the cache cost.
Keep the stable prefix byte-identical. Not semantically identical — identical. Whitespace, ordering of tool definitions, JSON key order all matter.
Deploying a system prompt change invalidates the cache for everyone. Expect a cost and latency bump after deploys, and don’t diagnose it as a regression.

This can conflict with the layout advice. If caching matters and something stable would ideally go late, you have a genuine trade-off — measure both.

Testing your layout

Position effects are model- and task-specific, so treat all of the above as a starting point.

The experiment is cheap:

Take a fixed evaluation set with known-correct answers.
Run your current layout, record quality.
Change one positional thing — passage order, instruction placement, history position.
Re-run, compare.
Keep what wins.

One trap worth avoiding: the passage containing the answer should sit in different positions across your test cases. If it’s always first, you’ll measure a layout that works for that arrangement and learn nothing about the middle — which is the thing you were worried about.

Re-run this when you change model. Positional behaviour is one of the things that varies most between models, and a layout tuned for one can be mediocre on another.

Summary

Ends are more reliable than the middle; the magnitude varies and is improving.
Stable content first — for framing and for caching.
Bulky retrieved content in the middle, labelled with source identifiers.
Critical instructions restated immediately before the query.
The user’s query last.
Never put volatile data in a cacheable prefix.
Test with the answer in varying positions, and retest on model change.

How much you can afford in each section is the budget; what to remove when it doesn’t fit is the cut order.