I Read Every Prompt Engineering Guide. Here Is What Actually Works.

Watch on YouTube ↗

There is a specific kind of prompt engineering advice that does not work: advice that is true for one model, on one task, for one person — packaged as universal principle. Most guides are full of it. "Be specific." "Use delimiters." "Say please." Some of these are fine. Some are neutral. A handful actually matter, and for identifiable mechanical reasons.

I've gone through the Anthropic and OpenAI official guides, the 26-principles paper from Bsharat et al., and enough practitioner write-ups to see which claims repeat across sources with genuine empirical backing. Here is the synthesis, organized by what we understand mechanistically versus what is empirically observed but unexplained — and what the evidence suggests is mostly noise.

What Chain-of-Thought Actually Does

Chain-of-thought prompting is the most studied technique in the field, and the mechanism is better understood than most people treat it. Adding "think step by step" to a prompt — or showing the model a few-shot example that includes intermediate reasoning steps — consistently improves performance on multi-step reasoning tasks. The reason is not that models "think better" when prompted to do so. It is that externalizing reasoning into the output sequence forces the model to produce intermediate tokens that constrain subsequent tokens.

When a model answers a multi-step arithmetic or logic problem directly, it generates the answer token in one forward pass informed by all prior context. There is no mechanism for checking internal consistency across steps that the model never stated. When the reasoning chain is in the output, each step is a real token that the model's subsequent generation must cohere with. Errors can compound, but they can also be interrupted — the chain makes the model's reasoning inspectable, and it makes incorrect premises visible rather than buried.

This matters practically: chain-of-thought gives the most reliable gains on tasks with objective intermediate steps — math, logic, structured extraction. It gives less consistent gains on tasks where the "steps" are vague (creative writing, open-ended summarization), because there is no checkable sequence of correct intermediate states.

Zero-shot chain-of-thought ("think step by step") works on capable models. Few-shot chain-of-thought — where you provide examples with explicit reasoning traces — works more broadly, because the model is pattern-matching to a demonstrated format rather than independently generating the structure.

Output Format Specification Is Not Trivial

Specifying the output format is cargo-culted as obvious, then under-applied in practice. "Return a JSON object" is not the same as defining the schema. Vague format instructions — "respond in a structured way" — produce inconsistent structure. Explicit schemas with field names, types, and examples produce far more reliable output.

The mechanism here is straightforward: format specification reduces the degrees of freedom in the output space. The model is not choosing whether to use a list or prose — it is filling in a defined structure. This is why output format specification is the highest-leverage single change for extraction and classification tasks.

For long or complex outputs, specifying format section-by-section matters more than a single upfront instruction. The model's adherence to a format specified at the start of a prompt degrades over long outputs, because later tokens are influenced less by the format instruction in the system prompt and more by local coherence in the output so far. Repeating format constraints or breaking output into sections using delimiters partially compensates.

Few-Shot Examples: Quality Over Quantity

The research on few-shot prompting converges on something counterintuitive: example quality and diversity matter more than count. Three examples that cover the range of edge cases outperform ten examples that are all similar instances of the easy case.

Why this works mechanistically: in-context learning works by activating distributions over tasks that the model learned during pretraining. A diverse, well-chosen set of examples signals the precise task distribution more accurately than a large set of homogeneous examples. Homogeneous examples can push the model toward a narrow pattern — the model generalizes from the examples it sees, not from the examples you meant to represent.

A few practical consequences. First, include examples of the hard or ambiguous cases, not just the clear ones — the easy cases are usually cases the model handles correctly without examples. Second, example order matters non-trivially, with recency having a modest effect; the last example before the user query has outsized influence. Third, incorrect examples are significantly more harmful than no examples — a wrong demonstration teaches the wrong pattern.

Negative Examples: What Not To Do

Telling the model what not to do is consistently underused. Research on the 26-principles framework specifically identifies this as one of the most reliable ways to reduce known failure modes. If a model consistently produces verbose answers, "do not use filler phrases or repeat the question back to the user" is more effective than "be concise." The negative constraint specifies the exact behavior to suppress rather than requiring the model to infer it from a positive description.

Negative constraints work because they are more precise. "Be accurate" is vague — it does not specify what inaccuracy looks like for your task. "Do not state confidence in a claim you cannot support with information provided" is precise and operable. The model can check compliance with the second instruction in a way it cannot with the first.

The practical pattern: after you have identified a failure mode in your model's output, write the negative constraint that directly prohibits that behavior. One sentence per failure mode. Add these to the system prompt section that addresses output quality.

Role Assignment: The Empirical Middle

Role assignment ("You are an expert in...") is one of the most discussed techniques and also one of the most oversimplified. The evidence is that role assignment works, but the effect varies substantially across models and tasks, and we do not have a fully satisfying mechanism.

The strongest hypothesis is that role assignment activates relevant distributions from pretraining. When you tell a model to respond as a senior security engineer, the tokens associated with that framing bring forward patterns from content written by security engineers — terminology, level of detail, cautious hedging on ambiguous cases. This is plausible and partially supported by how role framing shifts vocabulary and structure.

What this means practically: role assignment is most reliable when the role is specific and the domain is narrow. "You are an expert" adds almost nothing. "You are a compliance officer reviewing derivatives contracts for potential Dodd-Frank violations" specifies a domain, a task type, and a regulatory frame — each of which genuinely narrows the response distribution. The same logic applies: vague role framing is noise; precise role framing is signal.

On the empirically-observed-but-unexplained side: emotional framing ("this is critical to my career," "take a deep breath and think carefully") has measurable effects in studies. Why an LLM's output would change based on statements about the user's emotional state is not well understood from first principles. The training data hypothesis — that these phrasings appear before careful responses in training corpora — is plausible but unconfirmed. Treat emotional framing as a low-cost, uncertain bet.

Self-Consistency and Sampling

Self-consistency — generating multiple outputs and selecting by majority vote or synthesis — is a reliable method for improving accuracy on tasks with objective answers. It is also expensive. The tradeoff is direct: if you sample the same prompt five times and take the majority answer, you pay 5x the inference cost and get measurably more reliable answers on reasoning tasks.

Why it works: stochastic sampling means each output explores a different path through the model's output distribution. Averaging over paths cancels out errors that are locally coherent but globally inconsistent — the kind of error chain-of-thought alone cannot prevent. Self-consistency is most valuable when: the task has an objective correct answer, errors are not systematic (if the model has a consistent wrong belief, all samples will share it), and inference cost is acceptable.

A cheaper variant: ask the model to review its own answer before finalizing it. "Before giving your final answer, check your reasoning for errors" adds one step but costs less than full self-consistency. The effect is smaller but not negligible — it catches a subset of the errors that self-consistency would catch.

What Doesn't Generalize

Several widely-cited principles show inconsistent or task-specific effects.

Magic phrases. Adding "think step by step" to a prompt that already has a well-specified reasoning structure often does nothing. The phrase works when it introduces structure the model would not otherwise use. When the structure is already present, it is noise. The same applies to "answer in detail" — if the task requires a one-sentence answer, requesting detail produces worse output.

Prompt mirroring. Rephrasing the user's query back to them (as a confirmation step) is common in assistant applications. It adds latency without reliably improving accuracy. In production pipelines, skip it.

Length inflation. Instructing the model to "be thorough" or "provide a complete answer" reliably increases length. It does not reliably increase quality. Length and quality are not correlated. On extraction and classification tasks, longer outputs frequently introduce errors that weren't present in shorter responses.

Authority appeals. "Research shows that X" in a prompt does not improve the model's reasoning — the model cannot verify the claim and doesn't reason from cited authority the way a human expert would. State the actual constraint directly instead.

The Hierarchy When Output Is Poor

When a model's output quality is not meeting requirements, there is a practical debugging order that is more efficient than random prompt iteration.

Start with task clarity. Can you state, in one sentence, exactly what the correct output looks like? If not, the prompt almost certainly lacks the specificity to constrain the model toward it. Ambiguous tasks produce inconsistent outputs — not because the model is failing, but because you have specified multiple plausible correct answers.

Second, check context. Before revising the instruction, verify that the model has the information it needs to respond correctly. Many apparent prompt failures are context failures: the model is not hallucinating or misunderstanding — it is interpolating from insufficient information. This is the territory covered in context engineering, which I'd treat as the next layer up from prompt engineering: the work of constructing what the model knows before it produces any output.

Third, add output format specification. If the task is extraction, classification, or structured generation, a well-defined schema eliminates a large category of errors before you touch the instructions themselves.

Fourth, add a few-shot example of the correct output. One high-quality example of the right answer to a similar input is usually more valuable than three additional sentences of instruction.

Fifth, add chain-of-thought if the task has verifiable intermediate steps. If it does not, skip this — it adds tokens without adding constraint.

Sixth, if systematic failure modes persist, add explicit negative constraints targeting each failure mode directly.

This order is not arbitrary. It runs from mechanisms with the most consistent empirical support to mechanisms with narrower applicability. Most production prompt failures resolve at steps one through four.

The Stable Foundation

Attention mechanisms and in-context learning are stable enough that the principles above will remain relevant as capability improves. What changes across capability tiers is the threshold at which a technique becomes necessary. A weaker model requires chain-of-thought for tasks a stronger model handles without it. Role framing matters more on a model with weaker task-disambiguation than on one that reads task structure precisely.

The implication: as you move to more capable models, some prompt engineering work becomes unnecessary. But the underlying principles — task clarity, context quality, format specification, diverse examples, explicit negative constraints — remain active levers. They just move the performance ceiling rather than compensating for basic capability deficits.

Knowing which principles have mechanistic grounding versus which are empirically observed but opaque versus which are mostly cargo-culted is what separates prompt engineering that compounds over time from prompt engineering that produces undocumented magic strings no one can maintain.