Prompting & Prompt Engineering

A large language model defines a conditional probability distribution over the next token given all preceding tokens, $P_\theta(x_{t+1} \mid x_{1:t})$. A prompt is simply the prefix $x_{1:t}$ supplied by the user, and generation proceeds by repeatedly sampling or decoding from this distribution. Because the model's parameters $\theta$ are fixed at inference time, the only lever available to a practitioner is the choice of prefix — this is the entire design space of prompting. Prompt engineering is the systematic process of searching that design space: choosing instructions, exemplars, formatting, and ordering so that the resulting distribution places high probability on the desired continuation. Prompts are typically composed from a small vocabulary of building blocks. A system prompt (or preamble) sets persona, tone, and global constraints ("You are a terse technical reviewer"). An instruction states the task in imperative form ("Summarize the following article in three bullet points"). Exemplars (also called demonstrations or shots) are input–output pairs illustrating the task; a prompt with zero exemplars is zero-shot, with a handful is few-shot, and the special case of exactly one is one-shot. Finally, an output format constraint specifies the shape of the answer — JSON with named fields, a fixed number of items, a particular delimiter — which matters enormously when the output feeds into downstream code. The mechanism by which exemplars help is exactly In-Context Learning: the model, having seen sequences with similar structure during pretraining, treats the exemplars as evidence about which task distribution it is currently sampling from, and updates its implicit beliefs about output style, format, and even the mapping function itself, purely through attention over the context — no parameter update occurs. Prompt engineering is the practice of arranging this evidence — instructions and exemplars — so that the posterior over "what task is this" concentrates sharply and correctly, and so that the posterior over "what format should the answer take" matches what downstream consumers expect.

Constructing an effective prompt is best understood as an iterative pipeline: draft a template, instantiate it with real inputs, generate, evaluate against a rubric or held-out set, and refine. The pseudocode below captures the practical loop used in real prompt-engineering workflows, including the decision of whether to add exemplars, how to select and order them, and how to validate output format. ``` function ENGINEER_PROMPT(task_description, dev_examples, model, format_spec):

# dev_examples: labeled (input, output) pairs held out for evaluation

# format_spec: a validator function checking output structure (e.g. valid JSON) template <- DRAFT_ZERO_SHOT(task_description, format_spec) score, failures <- EVALUATE(template, dev_examples, model, format_spec) if score >= TARGET_THRESHOLD: return template

# Zero-shot insufficient: escalate to few-shot candidate_exemplars <- SELECT_EXEMPLARS(dev_examples, k = K_MIN) best_template <- template best_score <- score for k in [K_MIN, K_MIN + STEP, ..., K_MAX]: exemplars <- SELECT_EXEMPLARS(dev_examples, k) for ordering in ORDERINGS_TO_TRY(exemplars): trial <- BUILD_FEW_SHOT(task_description, ordering, format_spec) trial_score, trial_failures <- EVALUATE(trial, dev_examples, model, format_spec) if trial_score > best_score: best_score <- trial_score best_template <- trial if best_score < TARGET_THRESHOLD: best_template <- DIAGNOSE_AND_REPAIR(best_template, failures) best_score, failures <- EVALUATE(best_template, dev_examples, model, format_spec) return best_template

function BUILD_FEW_SHOT(task_description, exemplars, format_spec): prompt <- SYSTEM_HEADER(task_description, format_spec) for (x_i, y_i) in exemplars: prompt <- prompt + FORMAT_PAIR(x_i, y_i) prompt <- prompt + "Input: {NEW_INPUT}\nOutput:" return prompt

function EVALUATE(template, dev_examples, model, format_spec): correct <- 0 failures <- [] for (x, y_true) in dev_examples: filled <- SUBSTITUTE(template, x) y_pred <- model.GENERATE(filled) if not format_spec.VALID(y_pred): failures.append((x, y_pred, "format_violation")) continue if MATCHES(y_pred, y_true): correct <- correct + 1 else: failures.append((x, y_pred, "semantic_error")) return correct / len(dev_examples), failures ``` Three design decisions inside this loop deserve emphasis because they are exactly where prompting most often goes right or wrong.

Exemplar selection. `SELECT_EXEMPLARS` should choose examples that are representative of the distribution of real inputs, not just easy cases — a template validated only on simple inputs collapses on edge cases in production. A common strategy is stratified sampling across input difficulty and category, and, for retrieval-augmented few-shot prompting, choosing exemplars semantically close to the current input (a technique that foreshadows Retrieval-Augmented Generation, where the "exemplar" becomes retrieved supporting text rather than a fixed demonstration).

Exemplar ordering. Because attention is not permutation-invariant with respect to position (positional encodings and recency effects matter), the order of exemplars changes output quality even when the set is unchanged. Practitioners should treat ordering as a hyperparameter, trying at minimum a random ordering, a difficulty-ascending ordering, and a difficulty-descending ordering, and keeping whichever scores best on held-out data.

Output format enforcement. Downstream code rarely tolerates free text. Effective prompts specify format explicitly ("Respond with a single JSON object with keys `label` and `confidence`, and nothing else") and are evaluated with a strict validator, `format_spec.VALID`, since a semantically correct answer wrapped in the wrong syntax is a production failure regardless of its content. The figure below shows how a filled-in prompt is structured as it flows into the model, and how that same structure is the seed of the more advanced templates covered later.

Computational cost. Prompting introduces no additional training cost — this is its central practical appeal relative to fine-tuning. The cost lives entirely at inference time and is governed by sequence length. For a transformer decoder, a forward pass over a prompt of length $n$ tokens plus generation of $m$ output tokens costs

time for the attention computation (quadratic in total sequence length $L = n+m$, linear in model width $d$ per layer, summed over layers), and

space to store the key-value cache used during autoregressive decoding. Few-shot prompts with $k$ exemplars of average length $\ell$ add roughly $k\ell$ tokens to $n$, so cost grows quadratically in the number and length of exemplars — a direct efficiency tradeoff against the accuracy gains few-shot prompting provides. This is why prompt engineering explicitly searches for the minimal $k$ that reaches a target accuracy (the `EVALUATE` loop above increases $k$ only until `TARGET_THRESHOLD` is met) rather than maximizing exemplar count unconditionally.

Search cost of engineering itself. The `ENGINEER_PROMPT` procedure evaluates $O(|\text{template variants}| \times |\text{dev\_examples}|)$ model calls, since each candidate template (zero-shot draft, each $k$, each ordering) must be scored against the full development set to obtain a reliable estimate. This is the practical bottleneck of prompt engineering at scale: each iteration is one full evaluation pass, and with a large search space (many $k$ values, many orderings, many phrasings) the loop can require hundreds to thousands of model calls before converging — a real cost when the model is a paid API.

Correctness argument. There is no formal guarantee that any prompt achieves a target accuracy — prompting is an empirical, not provably-correct, procedure, and this must be stated honestly. What can be argued rigorously is a weaker, well-supported claim: under the in-context learning hypothesis, a pretrained model implicitly performs approximate Bayesian inference over a latent "task" variable $z$ conditioned on the prompt,

where $z$ ranges over task/format/style concepts seen during pretraining. Each token of instruction or each exemplar is additional evidence that narrows $P_\theta(z \mid \text{prompt})$ toward the intended task. This gives a principled reason why more relevant, more clearly formatted, and more correctly ordered exemplars tend to improve accuracy — they concentrate the posterior over $z$ — while also explaining why prompting is inherently variance-prone: if the pretraining distribution under-represents a task or format, no amount of prompt polishing can force $P_\theta(z \mid \text{prompt})$ to concentrate correctly, because the required evidence was never encoded in $\theta$ in the first place. This is the honest boundary of prompting's power: it steers a distribution that already exists, it cannot create capability the model never acquired during pretraining.

Tradeoffs, stated plainly.

Consider a concrete, non-trivial task: classifying customer support tickets into `billing`, `technical`, or `account_access`, with a strict requirement that the answer be valid JSON for downstream routing. This is deliberately not a symmetric two-class toy problem — three asymmetric classes surface real failure modes.

Zero-shot attempt. ``` Instruction: Classify the support ticket below into one of: billing, technical, account_access. Respond with only the category name. Ticket: "I was charged twice for my subscription this month and also can't log into the dashboard." ``` Model output: `billing` — but the ticket plausibly involves both billing and account access, and the zero-shot prompt gives no guidance on tie-breaking, and no format constraint, so a downstream JSON parser receives a bare word rather than the required object and fails.

Few-shot repair, iteration 1. Add two exemplars and an explicit format constraint, but keep exemplars easy (unambiguous single-category tickets): ``` System: You are a support-ticket classifier. Always output a single JSON object: {"category": ""}. If a ticket touches multiple categories, choose the PRIMARY complaint. Input: "My card was billed twice this month." Output: {"category": "billing"} Input: "The app crashes every time I open settings." Output: {"category": "technical"} Input: "I was charged twice for my subscription this month and also can't log into the dashboard." Output: ``` Model output: `{"category": "billing"}` — the JSON format is now correct, but on a held-out dev set of 40 tickets, ambiguous multi-category tickets (12 of the 40) are misclassified 50% of the time, because both exemplars were unambiguous and gave no evidence about the tie-breaking rule in practice.

Few-shot repair, iteration 2 — following `DIAGNOSE_AND_REPAIR`. Add one ambiguous exemplar that demonstrates the tie-breaking rule explicitly, and reorder so the ambiguous case comes last (closer to the query, exploiting recency): ``` Input: "My card was billed twice this month." Output: {"category": "billing"} Input: "The app crashes every time I open settings." Output: {"category": "technical"} Input: "I can't log in, and I noticed an extra charge too, but the login is what's blocking me right now." Output: {"category": "account_access"} Input: "I was charged twice for my subscription this month and also can't log into the dashboard." Output: ``` Model output: `{"category": "account_access"}` — matching the human label, because the login issue is the blocking problem. Re-running `EVALUATE` on the full 40-ticket dev set:

This progression illustrates the loop concretely: format failures are fixed by explicit format instructions and demonstrated JSON in exemplars; semantic failures on a specific input class are fixed not by more exemplars in general but by an exemplar that demonstrates the specific decision rule the model was missing, placed where recency helps most.

Prompting as a discipline emerged directly from empirical observations about large pretrained language models, rather than from a single theoretical derivation, which is part of why it remains as much craft as science.

Prompt engineering is the primary interface layer for essentially every production system built on a general-purpose LLM. Search engines and enterprise assistants use carefully engineered system prompts to constrain tone, prevent disallowed content, and force structured citations. Coding assistants embed few-shot exemplars of desired code style directly into the prompt so that generated completions match a project's conventions. Customer-support automation, exactly like the worked example above, depends on strict output-format prompting so that model output can be parsed and routed by downstream code without human review. Data-labeling pipelines use prompting to turn a frozen model into an on-demand annotator, with exemplar selection and evaluation loops nearly identical to `ENGINEER_PROMPT`. Prompting is also the foundation the next tier of techniques builds directly on top of. Chain-of-Thought Reasoning is, structurally, a prompt template that adds an instruction or exemplars demonstrating step-by-step reasoning before the final answer, exploiting the same in-context posterior-sharpening mechanism described above but targeting a reasoning process rather than just an output label. Retrieval-Augmented Generation automates and scales the exemplar/context-selection step: instead of a fixed set of hand-written exemplars, relevant text is retrieved per-query and injected into the prompt, turning `SELECT_EXEMPLARS` into a learned retrieval system. Tool Use & Function Calling extends output-format engineering further still, training or prompting models to emit structured function-call syntax that an external system executes and feeds back into the context — the same discipline of explicit schemas and strict validators seen in the JSON worked example, now applied to actions rather than labels.

Ask the tutor: How would you design a prompt template that reliably chooses between few-shot exemplars retrieved dynamically for each new input, rather than using a fixed set?