Wiki
Prompt Engineering
Prompt engineering techniques: role prompts, examples, structured output, evaluation, RAG context, and injection risks.
Related Wiki Pages
Prompt engineering shapes the input to an LLM. It names the role and task before the model answers. It also provides examples, retrieved context, and output constraints. In LLM applications, the prompt sits between the user task and the retrieved evidence. It also sits between model behavior and answer checks.
Prompt engineering is narrower than the whole LLM application. LLM Production Patterns covers serving, deployment, observability, and model choice. The release boundary for those decisions sits in LLM Deployment.
AI Tooling covers libraries and developer tools around prompts, while LLM tools covers the wider product stack. That stack includes model access, retrieval, evaluation, and review. It also includes agent orchestration and observability.
The prompt-engineering question is what the model sees and how the team constrains the answer. It also covers how teams test prompt changes and when wording alone stops helping.
The RAG vs Fine-Tuning comparison covers the boundary between better prompts, retrieved context, and model adaptation. Michael Taylor and James Phoenix’s Prompt Engineering for Generative AI catalogs the same role, example, and structured-output techniques as a practitioner reference. It also covers prompt-testing patterns across image and text generation.
Prompt Interface
Prompt engineering gives the model the right job, evidence, and target format. Role and objective come first. Examples, heuristics, audience-specific criteria, and expert review define what a good answer should satisfy. The person who used to do the work can help define the expected answer. [1]
In-context learning gives the same definition from the example side. Examples tell the model what should happen in a similar case. When a stronger model still misses the task, examples usually work better than a longer explanation. [2]
Prompt engineering belongs inside AI engineering even though it isn’t the whole system. Context engineering draws the wider boundary. Teams choose what to give the LLM instead of stuffing everything into the input. [3]
Machine translation is a narrow example of that interface work. Prompts can customize ChatGPT translation behavior. Quality control still has to sit around the model rather than trusting a fluent translation by default. Maria Sukhareva describes prompt-customized machine translation as useful, but still dependent on human expertise and quality control [4] [5].
The useful prompt isn’t only “translate this.” It can specify register, such as formal or informal plural. It can also ask for terminology consistency. A human translator still checks that the target text matches company language. They also check for safety-critical mistakes in manuals or technical content [4].
Boundaries and Tradeoffs
Prompt engineering can be the main task interface. Role and examples describe the work the model should do. Output schema and evaluation criteria describe what the answer must satisfy. [1]
It can also be one layer in a larger system. Retrieval, context selection, tests, and security controls all affect whether the answer can be trusted. Tool orchestration adds another failure point outside the prompt. [3]
The stopping point also differs by failure mode. When the model lacks current or source-grounded knowledge, Retrieval Augmented Generation is the stronger move. For specialized behavior or style, fine-tuning may be a better fit. Security failures need more than stronger wording. Validation and access control must enforce the boundary. [6] [7]
Role Prompts and Task Framing
Role prompts help when the role changes the answer criteria. A chief marketing officer campaign prompt becomes useful when the role comes with examples, heuristics, audience constraints, and judgment rules. The role isn’t valuable because it flatters the model. It’s valuable when it changes what the answer must satisfy. [1]
A prompt that only asks for YouTube chapters may produce plausible timestamps. Audience relevance and detailed review narrow the task. A pass/fail evaluator turns the loose role prompt into a reviewable task definition. [1] The team can then use LLM Evaluation Workflows to decide whether the prompt produced a usable result.
Roles can also hide weak task design. A prompt that says “be accurate” or “act as a secure assistant” doesn’t remove model nondeterminism. It also doesn’t remove provider updates or the need for validation. Endless prompt optimization can delay the system controls that actually reduce risk. [7]
At that point, teams should switch to evaluation or human review. Retrieval access control and classifiers may also be the next control. The Siemens chatbot examples show the boundary: better wording alone didn’t prevent knowledge-base extraction or hallucinated commitments.[8]
Structured Output and Examples
Teams handle structured output as a prompt-engineering problem, not just a formatting detail. The team can describe the JSON keys, or it can show a review and the expected JSON output. Examples often make the model follow the format even when the instruction is short. [2]
Early prompt checks can stay simple: teams can eyeball the output, then add structured outputs or regular expressions. They can also use string matching or cheaper models where the behavior is easy to assert. That keeps structured output close to testing instead of treating it as a formatting preference. [1]
Each extra example costs tokens and money. Teams use expected outputs for evaluation inputs to see when quality stops improving. That tells them when to stop adding examples. [2] Use LLM system design interview framing when a prompt question turns into a constraint question. Model calls, context size, and reliability belong to the same design decision.
Prompt Evaluation
Prompt evaluation starts with representative cases, not clever wording. A generator-evaluator setup can use one model to generate an output and another to check it. The final signal can be pass/fail with feedback, because a product usually needs to know whether the result can ship. [1]
Teams can start with manual review, but reliable software eventually needs examples that represent real user interactions. The test set should avoid overfitting to a few cases without wasting time and money. [1] For individual AI tools for personal productivity habits, teams can keep a smaller prompt-review set. Keep examples for the work you repeat, then compare new prompts against them before trusting the result.
Teams use failure analysis to decide whether more prompt work is worthwhile. Categorizing and ranking errors shows where the next fix belongs. If most failures come from retrieval, the next fix belongs in chunking, indexing, or source data. It doesn’t belong in another prompt rewrite. [1] Agent systems need the same style of evaluation because tool use and retrieval can fail outside the prompt. [3]
Public benchmarks test model capability. Product teams need custom datasets, mocked tools, integration tests, and outcome assertions. [3]
Compression, Caching, and Context Budget
Prompt size matters because every token can affect cost, latency, and answer quality. Prompt compression creates a shorter prompt that should preserve the same behavior. It’s an optimization topic, not a replacement for evaluation. A compressed prompt still needs the same expected-output checks as the original. [2]
Provider-side caching can help with repeated prompt prefixes. AI coding tools may reuse a shared codebase context with different user requests. Teams still need to verify the internal mechanism in provider documentation. At the product level, stable shared context can reduce repeated processing when many requests start with the same material. [2]
The context budget has a quality side because latency, cost, and noisy context all affect quality. Teams avoid filling large windows with material that creates garbage-in-garbage-out failures. [3] Prompt budgeting therefore connects to retrieval-augmented generation and production search evaluation. The prompt is only as useful as the context selected for it.
Context Engineering and RAG
Context engineering is the broader term for prompt work that selects and packages information for the model. Teams choose how to chunk it, which metadata to attach, and which wrapper helps the LLM use it. RAG is one example of context engineering, not a universal answer. [3]
RAG prompt structure injects relevant sections into a prompt and asks the model to answer from those documents. For sensitive tasks, a narrower flow retrieves the relevant section, summarizes or rephrases it, and keeps the answer grounded in that source. [6]
Chunking adds the practical constraint. A simple RAG bot can solve real support questions faster than an ambitious AI tutor, but chunking still depends on the source structure. [1]
A podcast transcript may work better by question-answer pair or speaker turn. A book needs a different strategy. The prompt can only use context that the retrieval system preserved.
Prompt Injection and Trust Boundaries
Prompt engineering also defines an attack surface, but instructions inside the prompt aren’t enough protection. In a Siemens chatbot challenge, 1,500 participants tried to bypass bot restrictions, and some extracted hidden knowledge-base content. [7]
A bot may have instructions not to reveal confidential information, and another layer may check the output. Users can still overload the prompt, use dense characters, craft API requests, or otherwise distract the model from the original restriction. [7] Prompt engineering therefore has a direct boundary with AI Red Teaming, Prompt Injection and Chatbot Risk Management, and Security. A secure system needs query analysis, output validation, retrieval controls, and human review where the risk warrants it.
Prompt injection is especially important for RAG because the model may receive instructions from user input and retrieved documents. If the system retrieves restricted or adversarial text, the prompt can include that text in the answer. The chatbot challenge shows that attackers can extract knowledge-base data. [7] The prompt template can state the rule, but access control and validation must enforce it.
Limits of Prompting
Prompt engineering has a clear stopping point. Fine-tuning is better for behavior, style, tone, and domain adaptation. Retrieval is better for changing knowledge and grounded facts. [6]
If a prompt repeatedly fails because the model lacks the right knowledge, the team should add retrieval. If it repeatedly fails because the model needs a specialized behavior, fine-tuning may be the better tool.
RAG is enough for some cases, but planning, tools, or agents are better fits when the system must coordinate multiple steps. Those systems need software-style tests, not only better instructions. [3] This links prompt engineering to Agent Engineering without turning every prompt problem into an agent problem.
Evolutionary algorithms can find useful prompt variations, but they’re computationally expensive. Use Game AI to LLM Agents for the broader bridge from game-AI search to agent behavior. [9]
Prompt iteration is useful, but teams should measure whether more iteration beats retrieval or fine-tuning. They should also compare it with tool design, human review, or a smaller product target.
Related Pages
Prompt engineering connects most directly to these adjacent wiki pages.