Asked across companies

English · Русский · AI Interview Atlas › Reference

Questions reported at two or more companies, strongest basis first.

LLM fundamentals

• Derive the memory required by a decoder's KV cache as batch size and context length grow.
  • Knowledge · Asked at: Amazon †, OpenAI † · ✍ Answer
• Compare greedy decoding, beam search and temperature, top-k and nucleus sampling; give a failure mode for each.
  • Knowledge · Asked at: Google and Google DeepMind †, Perplexity †

Inference, serving and cost

• Compare tensor, pipeline, data, sequence and expert parallelism for a large model deployment.
  • Knowledge · Asked at: Amazon †, Google and Google DeepMind †, Meta †
• Distinguish time to first token, time per output token, inter-token latency and throughput when comparing serving systems.
  • Knowledge · Asked at: Microsoft †, Perplexity † · ✍ Answer
• You are asked to reduce serving cost by an order of magnitude. Rank the levers and explain how you would test whether that target is achievable.
  • Applied scenario · Asked at: Amazon †, Microsoft † · ✍ Answer
• After a deployment, p99 latency doubles although model weights are unchanged. How do you isolate the cause?
  • Applied scenario · Asked at: Amazon †, Databricks †, OpenAI †, Perplexity † · ✍ Answer

RAG and retrieval

• When would you combine lexical and dense retrieval, and how would you merge their rankings?
  • Knowledge · Asked at: Microsoft †, Perplexity † · ✍ Answer
• Where should a cross-encoder reranker sit in a retrieval pipeline, and when does its quality gain justify latency?
  • Knowledge · Asked at: Microsoft †, Perplexity † · ✍ Answer
• Design retrieval that enforces the source system's access rights, including permission changes and shared caches.
  • System design · Asked at: Databricks †, Microsoft †, Palantir † · ✍ Answer

Agents, tools and protocols

• Design human approval for consequential agent actions, including how approval remains bound to the exact action being executed.
  • System design · Asked at: OpenAI †, Palantir † · ✍ Answer

Fine-tuning and post-training

• Choose between prompt changes, retrieval and finetuning for a failing AI feature; include data freshness, latency and total cost.
  • Applied scenario · Asked at: Databricks †, Microsoft †, OpenAI † · ✍ Answer

Evaluation and observability

• How would you measure unsupported claims in a deployed RAG application without treating every fluent answer as correct?
  • System design · Asked at: Anthropic †, OpenAI † · ✍ Answer

AI system design

• Design a knowledge assistant over ten million enterprise documents with per-user permissions and a continuously changing corpus.
  • System design · Asked at: Amazon †, Databricks †, Microsoft †, OpenAI † · ✍ Answer
• Design natural-language querying over a warehouse with thousands of tables, from schema selection to safe query execution.
  • System design · Asked at: Databricks †, Palantir † · ✍ Answer
• Design an LLM gateway with provider routing, failover, caching, rate limits and enforceable spending budgets.
  • System design · Asked at: Palantir †, Perplexity † · ✍ Answer
• Design a streaming chat service for hundreds of millions of users, including capacity, conversation storage and graceful overload behaviour.
  • System design · Asked at: Anthropic †, Google and Google DeepMind †, Meta †, OpenAI † · ✍ Answer

Practical coding

• Implement scaled dot-product attention with a causal mask and tests that would expose attention to future tokens.
  • Coding · Asked at: Amazon †, Anthropic †, Google and Google DeepMind † · ✍ Answer
• Implement a token-bucket limiter and explain what must change when several workers enforce the same limit.
  • Coding · Asked at: Anthropic †, OpenAI † · ✍ Answer
• Write an asynchronous API batch processor with bounded concurrency, retry jitter and isolated per-item failures.
  • Coding · Asked at: Anthropic †, Perplexity † · ✍ Answer