Agents, tools and protocols

English · Русский · AI Interview Atlas › Themes

Agent loops, tool design, memory, termination, human approval and protocols such as MCP.

On this page: Both tracks (4) · AI Engineering (15)

Both tracks

• When does splitting work among agents improve results, and when do coordination costs dominate?
  • Knowledge · ✍ Answer · Suggest a fix
  • Tests: Whether task independence and shared-state conflicts guide orchestration.
  • A strong answer covers:
    • Identify independent subtasks whose results can be combined without frequent shared-state updates.
    • Define delegation boundaries, completion criteria and a synthesis step that checks conflicting findings.
    • Compare with one agent at a comparable total budget; count duplicate work, coordination failures, cost and latency.
  • Read: How we built our multi-agent research system (Anthropic)
• How would you make an agent's production actions reversible where possible and auditable where reversal is impossible?
  • System design · Asked at: Palantir † · ✍ Answer · Suggest a fix
  • Tests: Whether compensating actions and durable evidence are planned before execution.
  • A strong answer covers:
    • Classify each tool action before release as reversible, compensable within a window, or irreversible.
    • Prefer native undo such as drafts, soft deletes and delayed sends; register a tested compensation for the rest.
    • Write intent, exact arguments and authoriser to an append-only log before acting; rehearse reconstruction from it.
  • Read: Designing robust and predictable APIs with idempotency (Stripe) · Compensating Transaction pattern (Microsoft Learn)
• Design human approval for consequential agent actions, including how approval remains bound to the exact action being executed.
  • System design · Asked at: OpenAI †, Palantir † · ✍ Answer · Suggest a fix
  • Tests: Whether approval controls are enforced outside the model.
  • A strong answer covers:
    • Preview the concrete action, target and consequences before approval.
    • Bind approval to validated arguments and recheck authorisation at execution.
    • Record the decision and execution result; expire approval after material changes.
  • Read: Building Effective AI Agents (Anthropic) · Designing robust and predictable APIs with idempotency (Stripe)
• Design an agent that creates and updates work orders in a live ERP without duplicating or silently corrupting business operations.
  • System design · Asked at: Palantir † · ✍ Answer · Suggest a fix
  • Tests: Whether business invariants survive uncertain model and network behaviour.
  • A strong answer covers:
    • Keep the model on proposals: a typed command validated against ERP business rules before any write.
    • Derive a stable business key for idempotent creation, and update with a version check, never last-write-wins.
    • Reconcile ambiguous outcomes before retrying, log proposal against effect, and test duplicate events and concurrent edits.
  • Read: Designing robust and predictable APIs with idempotency (Stripe)

AI Engineering

• What does interleaving reasoning with tool observations add to a language-model agent compared with reasoning alone?
  • Knowledge · ✍ Answer · Suggest a fix
  • Tests: Whether external feedback changes the agent's next action.
  • A strong answer covers:
    • Distinguish a planned action from an observation returned by the environment.
    • Use observations to update the next step and detect failed assumptions.
    • Discuss tool failures and the extra latency of repeated interaction.
  • Read: ReAct: Synergizing Reasoning and Acting in Language Models (Yao et al., arXiv)
• Design recovery from tool errors and timeouts, including cases where a timed-out call may already have changed external state.
  • System design · Asked at: OpenAI † · ✍ Answer · Suggest a fix
  • Tests: Whether retries preserve correctness as well as availability.
  • A strong answer covers:
    • Classify transient, permanent and ambiguous outcomes before retrying.
    • Use idempotency keys or reconcile external state after uncertain writes.
    • Bound attempts and deadlines; add backoff, jitter and an escalation path.
  • Read: Designing robust and predictable APIs with idempotency (Stripe)
• How do schema-constrained responses differ from function calls, and which component actually executes an action?
  • Knowledge · ✍ Answer · Suggest a fix
  • Tests: Whether valid syntax is distinguished from authorised execution.
  • A strong answer covers:
    • Separate constrained response formatting from a request to call a tool; neither executes or authorises an action by itself.
    • Validate completion status, schema and business rules, including refusal, truncation and unsupported evidence; syntactic validity is not factual correctness.
    • Keep execution, least privilege, confirmation and bounded recovery in application code; test a valid-looking but unauthorised request.
  • Read: Tool use with Claude (Claude Platform Docs) · Structured outputs (Anthropic)
• What does MCP standardise between an AI application and an external service that a model's function-call format does not?
  • Knowledge · Asked at: Microsoft † · ✍ Answer · Suggest a fix
  • Tests: Whether protocol transport and capabilities are separated from model decisions.
  • A strong answer covers:
    • Separate the model's choice of a call from how a client discovers and reaches a server.
    • Name what the protocol fixes: transport, capability discovery, and typed tools, resources and prompts.
    • Discuss the trust boundary: a connected server supplies both capabilities and untrusted content.
  • Read: Model Context Protocol specification (Model Context Protocol) · Security Best Practices (Model Context Protocol)
• How would you select, name and document tools so that a model chooses the right operation and arguments?
  • System design · Asked at: Anthropic † · ✍ Answer · Suggest a fix
  • Tests: Whether tool design reduces ambiguity and measurable misuse.
  • A strong answer covers:
    • Expose cohesive operations with unambiguous names and validated schemas.
    • Return relevant, bounded results and actionable error messages.
    • Evaluate tool selection and arguments on realistic tasks, then remove confusing overlap.
  • Read: Writing effective tools for AI agents—using AI agents (Anthropic)
• Design memory for an agent that works across many sessions: what is persisted, retrieved, revised and forgotten?
  • System design · Asked at: Anthropic † · ✍ Answer · Suggest a fix
  • Tests: Whether durable state is distinguished from transient context.
  • A strong answer covers:
    • Separate the working context of one session from state that must outlive it.
    • Write memory from confirmed outcomes, with provenance and a revision path, not from every utterance.
    • Retrieve by relevance to the current task and expire or supersede stale entries.
  • Read: Effective context engineering for AI agents (Anthropic)
• How should an agent decide whether to answer directly or obtain fresh information with a tool?
  • Knowledge · Suggest a fix
  • Tests: Whether uncertainty, freshness and tool cost influence the decision.
  • A strong answer covers:
    • Classify the question first: stable knowledge, time-sensitive facts, or private data the model cannot have.
    • Weigh the cost of a wrong direct answer against the tool's latency, price and failure risk.
    • Measure unnecessary and missed calls on labelled cases; force or forbid tools where policy requires.
  • Read: Tool use with Claude (Claude Platform Docs)
• Define completion and stopping conditions for an agent loop so it cannot spend indefinitely on an unfinished task.
  • System design · ✍ Answer · Suggest a fix
  • Tests: Whether progress, budgets and terminal states are explicit.
  • A strong answer covers:
    • Separate success, blocked, cancelled and budget-exhausted outcomes.
    • Track progress using observable state, with step, cost and time limits.
    • Checkpoint enough state to resume or hand off without repeating side effects.
  • Read: Building Effective AI Agents (Anthropic)
• A long-running agent confidently pursues the wrong objective. How do you locate where its task state diverged and recover?
  • Applied scenario · ✍ Answer · Suggest a fix
  • Tests: Whether diagnosis uses task state and observations rather than confidence in prose.
  • A strong answer covers:
    • Compare persisted task state and tool observations against the original goal, ignoring the agent's narrative.
    • Find the step where a summary, context compaction or misread observation replaced a constraint.
    • Resume from the last consistent checkpoint, and add periodic checks of progress against the written goal.
  • Read: Effective context engineering for AI agents (Anthropic)
• For a coding agent, how would you separate the model's contribution from the harness's contribution to reliable task completion?
  • System design · Asked at: Anthropic † · ✍ Answer · Suggest a fix
  • Tests: Whether controlled comparisons can identify the source of improvements.
  • A strong answer covers:
    • Define task success through independent behavioural checks, execution boundaries and resource budgets, not a model's assertion of completion.
    • Compare model and harness changes separately on the same tasks and repeat stochastic attempts; version context, tools and test environment.
    • Inspect traces for invalid edits, test weakening, environment failures and retries; report uncertainty and failures that an aggregate score hides.
  • Read: Demystifying evals for AI agents (Anthropic)
• Why might an enterprise agent operate on an ontology of business objects and actions instead of raw tables and documents?
  • Knowledge · Asked at: Palantir † · Suggest a fix
  • Tests: Whether semantic modelling is connected to valid operations and permissions.
  • A strong answer covers:
    • Explain that business objects carry meaning, relationships and identity that raw tables leave implicit.
    • Expose typed actions with preconditions and permissions instead of free-form queries and writes.
    • Weigh the modelling and upkeep cost, and test what happens when a task falls outside the ontology.
  • Read: Writing effective tools for AI agents—using AI agents (Anthropic)
• Build a customer-support agent for a fictional company: choose two of five proposed features, justify the choice and explain how you would measure the agent in production.
  • Applied scenario · Asked at: Sierra 🗣 · Suggest a fix
  • Tests: Whether you prioritise scope under a deadline and plan measurement for a working agent.
  • A strong answer covers:
    • Choose the two features by ticket volume, verifiable resolution and the risk of acting wrongly.
    • State what you cut and why, and keep escalation to a human working end to end.
    • Measure genuine resolution, repeat contacts, escalation quality and harmful actions, not conversations closed.
  • Read: Demystifying evals for AI agents (Anthropic)
• Design how a model plans and executes a long-running task through MCP tools while staying reliable within context-window limits.
  • System design · Asked at: Anthropic † · Suggest a fix
  • Tests: Whether you manage state, failures and context budget across a long tool-driven run.
  • A strong answer covers:
    • Keep the plan and progress in durable state outside the context, not only in the transcript.
    • Budget the context: paginate and trim tool results, compact history, fetch details only when needed.
    • Make tool steps resumable and idempotent, and test recovery from a server failure mid-run.
  • Read: Model Context Protocol specification (Model Context Protocol) · Security Best Practices (Model Context Protocol) · Effective context engineering for AI agents (Anthropic)
• What are a language model's generation parameters?
  • Knowledge · Asked at: Tochka ✅ · Suggest a fix
  • Tests: Whether you distinguish model generation settings from instructions about role, style or response structure.
  • A strong answer covers:
    • Name the decoding settings: temperature, top-p, top-k, maximum output tokens and stop sequences.
    • Separate them from prompt instructions on role, tone and format; an enforced output schema is an API setting.
    • Show each setting's effect on repeated samples, and check which ones a given API actually accepts.
  • Read: The Curious Case of Neural Text Degeneration (Holtzman et al., arXiv)
• An agent's context fills up over a long task. How would you decide what stays in the window, what is summarised and what moves out of it?
  • System design · 🧪 generated from job-posting themes · ✍ Answer · Suggest a fix
  • Tests: Whether you treat the context window as a budget with measurable costs rather than as space to fill.
  • A strong answer covers:
    • Separate what the next step needs from what only the record needs: keep the goal, the current plan and recent observations, and move finished detail to a store the agent can search.
    • Compact at boundaries you choose, such as a completed sub-task, and keep identifiers, open questions and decisions verbatim, because a later step reads those back.
    • Measure the cost: tokens and latency per step, and task success as the window fills; effective context is usually well short of the advertised limit.
  • Read: Effective context engineering for AI agents (Anthropic) · Lost in the Middle: How Language Models Use Long Contexts (Liu et al., arXiv) · RULER: What's the Real Context Size of Your Long-Context Language Models? (Hsieh et al., arXiv)

← RAG and retrieval · Fine-tuning and post-training →