Answers Are Cheap. Questions Are the Work.

Answers used to be expensive. You paged a senior engineer, waited for the architect to come back from lunch, or spent two days grinding through a vendor’s docs. The LLM changed that. A decent answer to almost any well-scoped technical question is now roughly free and roughly instant.

The bottleneck moved upstream, into the question itself. Ask a vague thing and you get a confident vague thing back, which is worse than no answer because it looks like progress. What matters now is framing the question well enough that the answer is worth having.

Five things help, in my experience, and none of them are prompt engineering tricks. They’re just the discipline of asking properly.

1. State the constraints

Most bad LLM output comes from missing constraints, not missing knowledge. The model will happily recommend Kafka when you have three services and one engineer, because you didn’t tell it you have three services and one engineer.

Constraints are the boring stuff: team size, budget ceiling, latency budget, existing stack, compliance boundary, who’s on call at 3am. A question like “should we use a vector database” is close to unanswerable. The same question with “we have about 200k documents, one backend engineer, already running Postgres, p95 latency needs to stay under 300ms, and nobody wants to operate a second datastore” almost answers itself.

Before you hit enter, try writing one line that starts with “given that we cannot…” and one that starts with “and we already have…”. If you can’t fill those in, you’re not ready to ask yet.

2. Supply the context the model cannot infer

The model has read a lot of public writing about software. It’s read nothing about your codebase, your last incident, or the political reason the previous team picked DynamoDB. That context is where the real decision lives.

Paste the relevant schema. Paste the actual error. Paste the section of the RFC that people are arguing about. If the CTO has a strong preference and that’s a real constraint on the decision, say so in the prompt. The model has no way to know that your “microservices migration” is actually four services behind an ALB and a Rails app that everybody still pretends is temporary.

This is also why “just use ChatGPT” as a company strategy underperforms. The value isn’t in the model itself, it’s in the context you feed it, and most orgs have that context locked in Slack threads and one person’s head. Anthropic’s work on contextual retrieval is a nice illustration at the infra layer: even with a strong model, retrieval quality craters once you strip out the surrounding document context.

3. Ask for the reasoning and the tradeoff, not the conclusion

If you ask “what should we pick,” you get a pick. If you ask “what are the two or three viable options, what does each one cost me in operational overhead, latency, and lock-in, and what would make me regret each choice in eighteen months,” you get something you can actually take to a decision meeting.

The conclusion is the least valuable part of an architectural answer. You’re the one who has to live with it, so you need the reasoning to pressure-test against things the model doesn’t know. A recommendation with no visible tradeoff is basically just a guess dressed up as advice.

I’d rather have a well-argued “it depends, and here’s the axis it depends on” than a confident “use Postgres.” Even when the answer is, in fact, use Postgres, which it very often is. Martin Fowler’s experience report on using ChatGPT for design work is interesting on this: the useful sessions were the ones where the engineer kept pushing on why, not the ones where they took the first suggestion.

4. Say what a good answer looks like before you ask

Tell the model, in the same message, what shape the response should take. Something like: “Give me three options in a table with columns for ops burden, monthly cost at 10M requests, and failure mode.” Or: “answer in two paragraphs, one for the case for, one for the case against, and don’t hedge.”

Specifying the output format does two useful things. It forces you to know what you actually want, which is often the hard part of the whole exercise. And it filters out the generic middle-of-the-road answer that shows up when you leave the shape open. Hamel Husain’s field notes on prompting push hard on being ruthlessly specific about the expected shape of the output, and it lines up with what I see day to day.

A small trick that works surprisingly well: write the first sentence of the answer you’re hoping for, then ask the model to continue it or argue against it. You learn more from the disagreement than the agreement.

5. Give it permission to say “I don’t know”

Models hallucinate partly because nothing in the conversation rewards admitting ignorance. The default posture is helpfulness, and a confident guess looks more helpful than a shrug, so a guess is what you get.

You can change the incentive by making the shrug an acceptable answer. Say it in the prompt, plainly: “If you’re not sure, say so instead of guessing. If you’re inferring rather than recalling, mark it. If this depends on a version or a detail I haven’t given you, ask me instead of assuming.”

This works better than it has any right to. The underlying problem isn’t really that the model invents things, it’s that the question you asked only had one socially acceptable shape of answer. Offer a second shape and the uncertain claims start arriving labelled as uncertain, which is usually all you needed.

For anything you’d actually act on, like a config flag, an API signature, or version-specific behaviour, add “flag anything you’re not confident about, and tell me what you’d check to confirm it.” The second half is the useful part. It turns an unverifiable claim into a next step you can take in thirty seconds.

The shift

None of this is new. Good senior engineers have always asked better questions than junior ones, and that’s most of what “senior” actually means in practice. The LLM just made the gap visible and expensive. If two engineers have the same tool and one gets useful output in a single round while the other is on their fifth follow-up, the model isn’t the difference between them.

So the next time you’re about to type a question into a chat window, spend thirty seconds on the constraints, the context, and the shape of the answer you want. The thirty seconds pays for itself before you finish reading the reply.