On prompting, part 1 of 3

Prompt Engineering Is Finding the Seams

Break the task into its parts, find where each one is missing information, and close those gaps in the layer that owns them.

In 2024, when GPT-4o and Claude 3.5 Sonnet came out within about a month of each other, prompt engineering looked like it would become the next big discipline. In some ways it was how most of us learned AI for the first time. In others, it was us trying to prompt our way out of a model that felt like magic but wasn’t quite intelligent yet. By many accounts that changed in November 2025, when a run of releases pushed the models over what Simon Willison called “an invisible capability line.”

A lot of the early tricks didn’t survive that. Telling the model it was a world-class expert, offering it a tip, writing in all caps: the newer models mostly don’t need any of it. But prompt engineering still has its legitimacy. On complex evaluation tasks and complex workflows, a model can’t do well at something the task never defined. So the job is to break the task into its parts and go through each one looking for missing information: instructions that aren’t well defined, judgments with no bar to reach, outputs with no rubric, and decisions nobody said the model was allowed to make. That missing information is where hallucinations come from.

I think of those gaps as seams, and the goal is to reduce the ambiguity at each one. Finding them is about half the work. The other half, and the part I think matters most at a staff or principal level, is deciding where each fix should live, because most of them shouldn’t live in the prompt.

Wording Was Never the Lever

There’s good research on why clever wording was a dead end. In a 2023 study, researchers changed only the formatting of few-shot prompts, things like separators, casing and spacing, and measured differences of up to 76 accuracy points on LLaMA-2-13B. Same meaning, very different answers.

Current models are much less sensitive than LLaMA-2 was, but I still take the lesson seriously. If a system only works because a sentence is phrased a certain way, you can’t explain why it works, and you’ll probably lose it the next time you upgrade the model. What carries over between model versions is meaning. A term with a precise definition means the same thing to every model, and so does a threshold. That’s the work worth doing.

Analyzing Your Task

The first step is analyzing the task and finding its seams. Almost no real task is a single step. Usually it’s a chain: read something, look something up, make a decision, produce something, check it. Each step takes the previous step’s output as its input, and each has its own idea of what doing it well means.

A seam is any point in that chain where the task leaves a decision to the model that nobody on the team actually made. Inside a step, it’s an instruction with something missing. Between steps, it’s a handoff where nobody said what the next step needs. A person on your team would fill the gap with context they picked up along the way. The model fills it with its best guess.

That guessing is how models are built, so you can’t prompt it away. Researchers at OpenAI and Georgia Tech argue that models hallucinate because training and evaluation reward guessing over admitting uncertainty: “language models are optimized to be good test-takers, and guessing when uncertain improves test performance.” Give a model an ambiguous step and it will usually guess, and it won’t tell you it guessed.

So I break the task down and go through it one step at a time, asking the same questions at each one:

Is every term defined? Plenty of words sound precise and aren’t: “urgent,” “active customer,” “revenue,” “the last concert.”
Is there a bar? “Is this adequate?” Adequate compared to what? The same goes for output. How does the step know its work is good enough to pass on?
Is there a quality rubric? What does good output look like for this step? Without one, the model makes up its own standard, and a slightly different one every run.
Can it see what it needs? Today’s date, the event calendar, which four product IDs count as a hot dog. If the step depends on a fact the model can’t see, it will assume one.
Does it have the authority? What is this step allowed to decide, promise or change, and which source wins when two disagree? If you don’t set limits, the model won’t assume any.
What happens when it can’t tell? If nothing says, the model will produce an answer anyway.

To make that concrete, take prior authorization adjudication. A provider asks a health plan to approve a service before it’s performed, and the plan decides whether it’s medically necessary. It makes a good example because the stakes are real and a lot of the rules are written down. Under CMS’s prior authorization rule, payers it covers have to decide urgent requests within 72 hours and standard ones within seven calendar days, and starting in 2026 they have to give a specific reason for every denial.

Here’s the task the way it often gets written:

Review this prior authorization request and decide whether to approve it.

That looks like one instruction, but it’s at least six steps, and each one has a seam:

Step	What’s missing	What the model will likely do
Intake	The service, diagnosis and member, as codes	Guess the procedure code from the fax
Eligibility and coverage	Whether the member was covered on the date of service, and whether the service needs authorization at all	Assume both
Choosing criteria	Which policy applies, and which source wins if two disagree	Use criteria it remembers from training
Gathering evidence	The clinical notes, in order, with dates	Skim eighty pages of scans and go with its overall impression
Deciding	A bar for when criteria are met, and who is allowed to deny	Deny the request itself
Writing the notice	The specific reason for the decision and the policy it’s based on	Write a generic explanation without either

The handoffs between those steps matter just as much. If intake gets the procedure code wrong, every later step evaluates the wrong service. If the criteria step picks the wrong policy, the evidence gets measured against the wrong bar. A gap that isn’t closed early carries into every step after it, which is how PwC’s pipeline failed in our BI rebuild.

In prior auth, some of the authority questions are already answered by regulation. CMS requires Medicare Advantage plans to follow Traditional Medicare’s coverage rules, meaning its national and local coverage determinations, and to use their own internal criteria only “when coverage criteria are not fully established.” That settles which source wins. And before a plan issues a denial based on medical necessity, the decision “must be reviewed by a physician or other appropriate health care professional.” A model can help recommend an approval, but a denial has to go to a clinician.

None of these are problems you fix with better writing. Each one is a decision someone on the team has to make, and you can’t hand those decisions to the model. The work is making them, one seam at a time.

Break Down Each Requirement

In prior auth, most of the seams are in the criteria. A medical policy reads like a single judgment, but it’s really several requirements bundled together. Here’s a simplified policy for a lumbar spine MRI:

A lumbar MRI is medically necessary for low back pain after six weeks of conservative therapy without improvement, or sooner when a red flag is present, such as a progressive neurological deficit, suspected cancer, suspected infection or symptoms of cauda equina syndrome.

If you ask a model “does this patient meet the policy?” you’re asking about half a dozen questions at once and getting one answer back. When it’s wrong, you can’t tell which part it got wrong, and neither can the clinician who has to review it. So break the policy into requirements that can each be checked on their own:

R1  Low back pain is documented as the reason for the request.
R2  Conservative therapy (physical therapy, medication, or both) is documented.
R3  That therapy started at least 42 days before the request date.
R4  The notes say the pain persisted or worsened despite it.
R5  A red flag is documented: progressive neurological deficit, suspected
    cancer, suspected infection, or cauda equina symptoms.

The policy is met when R1 through R4 are all MET, or R1 and R5 are MET.

Each requirement then gets its own outcome: MET, NOT MET or NOT DOCUMENTED. The third one is the most important, because missing evidence and failed criteria are different situations. If the notes don’t mention physical therapy, the right move is to pend the request and ask for the records. If you treat “not documented” as “not met,” you’ll deny a patient who did eight weeks of physical therapy just because those notes weren’t in the fax.

Prepare the Evidence First

The other source of seams is the documents themselves. Prior auth evidence arrives as faxes, scanned PDFs and EHR exports. That’s often dozens of pages, out of order, with a lot of it copied forward from earlier visits. If you give that to a model as-is, alongside the rubric, you’ve created new questions it has to guess at. Which note is the most recent? Is a given date the visit or the date the record was printed? When an old history says “PT x6 wks,” was that six weeks this spring or three years ago?

So I prepare the documents before anything evaluates them:

Extract the text. Run OCR on the scans, and keep the page numbers so any citation can point back to its page.
Split by encounter. Make one document per visit, each with its date of service.
Normalize. Use one date format, and turn therapies and medications into structured facts: what it was, when it started and how long it lasted.
Remove copied-forward text. The same history paragraph can show up on twenty pages. Keep it once, at the visit where it first appeared.
Map evidence to requirements. For each requirement, pull out the passages that could support or contradict it.

I use plain code wherever possible, for OCR, date parsing and splitting, and a model only where it’s needed, like pulling facts out of free-text notes. Then I check the model’s extractions in code too. For example, an extracted date has to actually appear on the page it cites.

After that, each requirement is evaluated against its own prepared evidence, one at a time. Instead of reading eighty pages and forming a general impression, where attention gets diluted as context grows, the model answers one narrow question about a handful of short, dated passages. Models are much more reliable at that kind of question. In my experience, running preprocessed documents against requirements that have been broken down is where most of the quality comes from.

Instructions

Once you’ve found the seams, you close them. Each kind of seam has its own fix:

Define terms as conditions someone could check, not adjectives.
Set a bar. Anthropic’s guide to defining success criteria is a good model. Its bad example is “the model should classify sentiments well,” and its good example names the metric, the threshold and the test set. The same idea applies inside a prompt.
Write a quality rubric. Describe each possible outcome, and require every decision to cite the evidence behind it.
Define good enough. Output passes when it meets every requirement on the rubric, not when it reads well.
Spell out authority. Say what the model can decide on its own, what it can only recommend, and which source wins when two disagree.
Allow “I can’t tell.” “Not enough information” should be an answer the model is allowed to give.

Here’s what that looks like for one requirement:

Requirement R3: conservative therapy started at least 42 days before the
request date of 2026-09-14.

Evidence: the passages below, each with its page and date of service.

Answer exactly one:
MET             Quote the passage, with its page and date.
NOT MET         Quote the passage that shows it wasn't.
NOT DOCUMENTED  Nothing below speaks to it. Name what's missing.

Don't infer therapy that isn't written down.

And for the decision step, which is mostly about authority:

Recommend APPROVE when the policy's rule is satisfied.
If any requirement is NOT DOCUMENTED, recommend PEND and list what to request.
You may not deny. Every other case goes to a clinical reviewer, with each
requirement's outcome and citations attached.

This also takes care of the denial notice. The specific reason CMS requires can be assembled from the requirements that weren’t met and the passages cited for them, instead of having a model write it from scratch.

These prompts aren’t better because they’re longer. They’re better because a person made every decision in them ahead of time.

Rubrics matter even more when a model is the one grading. LLM-as-a-judge can work well. In the MT-Bench study, strong judge models agreed with human preferences more than 80% of the time, which is about as often as humans agree with each other. The same paper also documents position bias, verbosity bias and self-enhancement bias, where a model favors answers like its own. Without a rubric, a judge tends to favor the longer answer or the one it read first. Defined outcomes and required evidence go a long way toward correcting for that.

Where the Fix Lives

This is where I think the architecture work really starts. Look back at those fixes and ask who owns each one.

The MRI policy belongs to the clinical policy team. Eligibility belongs to enrollment. The 72-hour deadline and the denial rules belong to compliance. If you paste all of that into a prompt, you’ve copied three teams’ decisions into a file none of them will ever look at. When clinical policy updates the MRI criteria, your prompt is out of date, and nothing will tell you.

In my experience that’s the bigger risk with prompt engineering, more than wording. Definitions get duplicated across prompts, and every copy drifts on its own schedule.

So I sort the fixes by who owns them and how often they change:

Business definitions change when the business changes. They belong with the data team, in one place that every consumer reads from.
Domain guidance and policy, like which criteria apply, which queries to avoid and what the model must never decide, change as the domain changes. They belong with the team that owns that domain, in a shared library where each requirement is defined once.
The task itself, meaning the bar, the rubric and the output format, changes when the job changes. That’s the only part that belongs in the prompt.

Sorted this way, prompts get much shorter, because they’re no longer carrying everyone else’s decisions.

Semantic Layers

Closing the seams reduces hallucinations in the task. The next problem is the data itself. Even with a well-defined task, a model looking at a slice of data doesn’t know the story behind it: what a number includes and leaves out, why a column is empty for three weeks, or which of two similar fields finance actually reports on. Without that context, the model will explain the numbers anyway, and the explanation will sound right.

That context is what a semantic layer is for, and the tool already exists. It organizes your data into layers: raw tables at the bottom, governed definitions and their context on top of them, and every consumer reading the same thing, whether it’s a dashboard, an embedded report or an agent.

Cube models this as cubes, measures and dimensions, with views on top that users query, and describes the result as “the shared context for AI agents, BI dashboards, and embedded analytics.” Here’s the hot dog problem from our BI agent, written as a definition, with the context behind the slice written next to it:

cubes:
  - name: concession_sales
    sql_table: concessions.sales
    measures:
      - name: hot_dogs_sold
        description: Units sold of every hot dog item, including the stadium dog, Polish sausage, kosher dog and chili dog. Excludes suite catering, which is billed separately.
        sql: quantity
        type: sum
        filters:
          - sql: "{CUBE}.category = 'hot_dog'"
    dimensions:
      - name: stand
        description: The concession stand. Stand 114 was closed for renovation from January through June, so its zero sales in that period aren't a drop in demand.
        sql: stand_id
        type: string

The examples are made up, but the pattern isn’t. The descriptions travel with the data, so when an agent slices sales by stand, it gets the reason behind the gap along with the number.

In our agent, the model worked out what “hot dogs” meant at runtime, every time, with a vector search and its own judgment. In a semantic layer it’s a definition the concessions team owns and writes once, along with what anyone looking at it should know, and every consumer gets the same answer.

The dbt Semantic Layer makes a similar case. Moving metric definitions into the modeling layer means that “if a metric definition changes in dbt, it’s refreshed everywhere it’s invoked.” Its docs now also say that AI tools like Claude and ChatGPT can connect through dbt’s MCP server, “so their answers use your governed metrics instead of guessing at raw tables.” That’s close to the argument I’m making here.

Snowflake’s semantic views put the same idea inside the warehouse. You define logical tables for your business entities, with facts, dimensions and metrics on them, and store the whole thing as a schema-level object. Snowflake’s own agents read that definition before they write SQL against the physical tables. Whichever tool you use, the point is the same: the model reads definitions and context that a person wrote down once, instead of working them out from raw tables.

Skills Sit Above It

A semantic layer can tell you what revenue is. It can’t tell you that finance means revenue net of refunds when they bring it up in a meeting, or that one particular query looks right but times out at the end of every month. That’s domain guidance, the second layer, and it’s where skills come in.

A skill is a folder with a SKILL.md file in it, in a format Anthropic published as an open standard. The agent loads only each skill’s name and description at startup, about a hundred tokens apiece, and reads the full file when a task matches. For prior auth, one might look like this:

---
name: lumbar-mri-prior-auth
description: Evaluates prior authorization requests for lumbar spine MRI
  against the plan's medical policy. Use when a request includes CPT 72148,
  72149 or 72158.
---

# Lumbar MRI prior authorization

Evaluate each requirement in [the policy](references/requirements.md) on its
own, against the prepared evidence for that requirement only.

## Gotchas

- Therapy dates come from the visit notes, not the referral. A referral
  date is when therapy was ordered, not when it started.
- Copied-forward history repeats old therapy. Use the visit where it first
  appears.
- A missing record is NOT DOCUMENTED, never NOT MET.
- Never recommend a denial. Send anything that isn't an approval or a pend
  to a clinical reviewer.

The requirements live in their own file, which the clinical policy team owns and updates. The skill holds the things a new reviewer would learn in their first month: where the dates actually come from, what tends to be misleading, and what the model is never allowed to decide.

In our BI agent we wrote that layer as a Markdown skill for each area of the business, and each skill listed the root fields it covered. A drift check compared those lists against the GraphQL schema on every change, so a skill couldn’t keep describing a field that had been removed. I’d argue that check matters as much as the skills. Without some way to verify a layer, it’s really just a longer prompt.

The layers each answer a different question. The semantic layer answers what something means. The skill answers how to use it safely. The prompt answers what we’re doing right now.

Treat It Like an Interface

With those layers in place, a prompt starts to look like an interface between your system and a component that doesn’t share your context, and I think it should be designed like one.

Version all of it. The prompt, the skills and the semantic model can each change an answer.
Use evals as the contract. Every seam you close becomes a test case. When a definition changes, rerun the suite and see what changed before users do.
Give every layer an owner. If no one would think to update a definition, it’s going to go stale.
Keep decisions out of the model. People make the thresholds, permissions and definitions, code enforces them, and the model applies them.

Takeaways

Prompt engineering didn’t go away in November 2025. The tricks did, and the engineering part is still here.

If I were starting a complex evaluation or workflow today, I’d spend most of my time before writing the prompt. I’d break the task into steps and go through every seam: undefined terms, missing bars, missing quality rubrics, inputs the model can’t see, unclear authority and no plan for when it can’t tell. I’d prepare the documents before anything evaluates them, and break each requirement out so it can be checked on its own. Then I’d put each fix in the layer that owns it. Business definitions, and the context behind each slice of the data, go in a semantic layer, domain guidance goes in skills with a drift check, and only the task goes in the prompt.