A client wants a model that follows their specific document formatting conventions and answers from their knowledge base. What do you recommend?

What the interviewer is testing

Whether you know what each technique actually does, rather than treating them as competing options.

The distinction

Fine-tuning teaches behaviour, format and style. It shapes how the model responds — structure, tone, conventions, output schema adherence. It is the right tool for "always produce this document structure with these section conventions".

RAG supplies facts. It puts the relevant knowledge in the context at query time.

Fine-tuning is a poor way to teach facts: knowledge is diffused into weights with no attribution, no freshness path, and no way to apply permissions.

Why RAG must handle the knowledge

Freshness. Updating RAG is a re-index of the changed documents. Updating a fine-tune is a training run and an evaluation cycle, repeated whenever the knowledge base changes.

Permissions. This is what settles it for enterprise content. RAG can filter retrieval by the requesting user's permissions at query time. A fine-tuned model has absorbed everything into its weights and cannot un-know a document for a particular user. There is no fix short of a model per permission set.

Attribution. RAG can cite the source passage, which is what makes an answer verifiable and is usually a hard requirement for internal knowledge tools.

The sequencing I would recommend

Start with prompting plus RAG. Careful prompting with few-shot examples achieves a great deal of format adherence, and structured output constraints handle the rest. Build the evaluation set at the same time.

Fine-tune only if measurement shows prompting is insufficient — persistent format deviations, or a prompt so long that the token cost of the examples exceeds the cost of fine-tuning. Then fine-tune on formatting examples, not on knowledge.

What a strong answer adds

The operational cost of fine-tuning that is usually omitted: a training and evaluation pipeline, model versioning, and re-tuning whenever the base model is updated or deprecated by the provider. That is an ongoing obligation, not a one-off.

Common weak answers

Fine-tuning on the document corpus, which fails on freshness, permissions and attribution. Prompt-only, when a genuine format requirement warrants tuning.