Production AI engineering: eight-week practice route¶
English · Русский · Learning path · Engineering
This route helps experienced developers explain the design and operation of a bounded service around a model. It sits inside AI Engineering alongside AI-assisted coding. The existing eight-week learning path remains the broad foundation, including attention, training and specialisms. Here the emphasis is contracts, evaluation and operational judgement.
Plan eight weeks at 6–8 hours per week, an editorial estimate rather than a measured preparation time. Prerequisites are familiarity with small services, HTTP failures, asynchronous work, databases and held-out evaluation. Use the baseline learning path to close those gaps. The current route requires questions, reading, explanations and checklists. Python labs, service implementation and an AI evaluator are deferred by the maintainer's current scope. Completion does not certify production readiness or predict hiring.
One conceptual case throughout¶
Consider an assistant for an invented equipment-maintenance catalogue. It would answer from versioned manuals and propose a maintenance ticket. Two fictional user groups have different document access. Reason about answerable, unsupported and forbidden requests; no dataset creation, model account, API key or running service is required. These editorial follow-ups are outside the published and radar-generated bank.
Keep the proposed boundaries explicit: no action merely because a model requests it, no answer from forbidden documents, an insufficient-evidence response, a release owner and a per-request time, usage and retry budget. Separate what a design promises from what only an implemented test could establish. Read the primary materials attached to the linked questions, then use the written answers as one defensible approach rather than a script.
Eight modules¶
Reserve 1–1½ hours for reading, 3–4 hours for question analysis and a short decision note, 1 hour for oral explanation and 1–1½ hours for feedback and delayed retries. Evaluation and access control begin in the first module.
| Week | Questions and reading | Explanation checklist and failure probe |
|---|---|---|
| 1 — task and baseline | What user outcome justifies AI? What can manual lookup or lexical search already do? Read system design, fundamentals and document intelligence. | State users, allowed documents, success, critical error and a reason to stop. Distinguish answerable, missing-evidence and denied-access requests. A million processed items do not establish useful or correct outcomes. |
| 2 — models and contracts | How would you compare configurations under the same task and budget? Who validates and executes a tool request? Read structured outputs and method choice. | Explain valid, malformed, refused and truncated responses; unknown evidence IDs and unauthorised but schema-valid calls. Name recovery limits and the visible terminal failure. |
| 3 — evaluation | How do you assess a task without one reference answer? Read scarce labels, agent evaluation and production actions. | Separate retrieval, answer, contract and action failures. Propose a held-out split, repeated attempts and expert adjudication. Explain why synthetic cases cannot establish real failure frequency and an uncalibrated model judge cannot approve release. |
| 4 — retrieval and context | When is added retrieval complexity justified? Read RAG and index freshness. | Compare lexical search with one alternative using proposed relevance, support and latency checks. Trace an exact identifier, stale manual, deletion and revoked access through retrieval, caches and output. An atomic alias switch alone does not enforce authority. |
| 5 — tools and execution | How does a bounded workflow stop and recover? Read retries, goal drift and MCP security. | Describe permitted state transitions and outcomes for unknown tool, malformed arguments, denied permission, timeout and exhausted steps. A timeout after a write requires reconciliation by operation identity before retry. |
| 6 — operations and cost | What would a trace and rollback need? Read inference economics, configuration rollback and SRE monitoring. | Name versions, duration, usage, status and error category; avoid unnecessary sensitive content. Include failed attempts in cost per success. Explain timeout, concurrency bound, fallback and a dependency that could invalidate rollback. Label estimates. |
| 7 — security and release | Which input can influence a plan without gaining authority? Read OWASP prompt injection, security and release gates. | Trace an injected manual requesting another group's data or a tool action. Put permission checks in the application. A critical access failure blocks release even when average quality improves; name the owner, stop condition and rollback action. |
| 8 — capstone discussion | Defend baseline, selected design and unresolved evidence using applied scenarios. Halve the budget, remove a provider or revoke a document. | Present an architecture sketch, proposed evaluation and release checklist. Explain what must be rechecked, one rejected alternative and the weakest assumption. The capstone is a design defence; implementation remains deferred. |
Four distinctions to keep explicit¶
Format and meaning. Schema conformance does not establish factual accuracy or business validity. Provider behaviour is version-specific: Claude's documentation lists refusal, token-limit and enum-case exceptions. Validate completion and application rules; do not retry indefinitely.
Repeatable checks and variable outcomes. A fixed dataset does not make model outputs or graders deterministic. Anthropic's evaluation account discusses grader types and calibration. Explain repetitions, disagreement and failure severity; an average gain cannot cancel a permission leak.
Filtering and authority. Input sanitisation alone is not a security boundary. OWASP describes controls including least privilege and separation of external content. Authorisation belongs in application code and must cover retrieval, tools, caches and logs.
Protocol and trust. MCP standardises integration; it does not establish trust in a server or its tools. The dated security guidance covers token audience, consent, SSRF and local-server risks. Name who grants authority, its resource scope and revocation. A protocol-only diagram is insufficient.
Completion and limits¶
Use the existing 0–3 rubric. Each module's explanation and delayed attempt need at least 2: an independently explained mechanism, assumptions, trade-off and relevant check. A 3 also handles a changed constraint and failure case. Scores 0–1 require review; a critical correctness or access error blocks the gate regardless of average score. Retry after roughly two days and the following week, recording date, feedback and the next weak point.
You can complete the current route without running a model. That demonstrates reasoning, not measured answer quality, latency, refusal behaviour or injection resistance. A future implementation could use synthetic data and deterministic stubs without API keys, but even passing stub tests would establish control flow only, not live-model quality. Do not claim unperformed checks. See Engineering research for provenance and access limits, and the expansion plan for deferred implementation.