AI PROJECT ONBOARDING Shared vocabulary turns AI ambition into sound project choices Key concepts for understanding LLMs and AI agents Strategy and engineering enablement · October 2026
AI project onboarding · Shared vocabulary 2 THE WHOLE SYSTEM A successful AI project designs the full system, not only the model Decision requested: Use this vocabulary to frame scope, owners, evidence, and trade-offs before selecting a solution. 1 Model capability Select a model and define what it can see in each interaction. 2 Grounded knowledge Retrieve trusted, current enterprise information rather than relying on model memory alone. 3 Controlled action Give the model narrow, observable tools instead of unrestricted system access. 4 Measurement and control Evaluate quality, enforce guardrails, and monitor cost and latency in production. Source: OpenAI — Tokens and tokenization ; Lewis et al. — Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks ; OpenAI — Function calling guide ; NIST — AI Risk Management Framework [Sources]
- OpenAI — Tokens and tokenization — OpenAI — https://platform.openai.com/docs/concepts/tokens
- Lewis et al. — Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — arXiv — https://arxiv.org/abs/2005.11401
- OpenAI — Function calling guide — OpenAI — https://platform.openai.com/docs/guides/function-calling
- NIST — AI Risk Management Framework — NIST — https://www.nist.gov/itl/ai-risk-management-framework
AI project onboarding · Shared vocabulary 3 MODEL FUNDAMENTALS Tokens and the context window set what the model can process in one interaction Tokens are model-readable pieces of text The context window is the model's working set Context is a product design constraint Text is split into tokens before processing; tokenization varies by model and language. Instructions, conversation history, retrieved knowledge, and output all compete for the same finite window. Decide what to include, summarize, retrieve, or omit so the model sees the relevant evidence. Source: OpenAI — Tokens and tokenization [Sources]
- OpenAI — Tokens and tokenization — OpenAI — https://platform.openai.com/docs/concepts/tokens
AI project onboarding · Shared vocabulary 4 MODEL LIFECYCLE Training builds a model; inference applies it to each request Training changes model parameters Inference applies the trained model The distinction changes project choices The learning process adjusts parameters from data to create general capabilities. Each request combines an input and context with the existing model to produce an output. Most enterprise work improves inference through prompts, retrieval, tools, and controls rather than training a model. Source: Vaswani et al. — Attention Is All You Need ; OpenAI — Fine-tuning guide [Sources]
- Vaswani et al. — Attention Is All You Need — arXiv — https://arxiv.org/abs/1706.03762
- OpenAI — Fine-tuning guide — OpenAI — https://platform.openai.com/docs/guides/fine-tuning
AI project onboarding · Shared vocabulary 5 GROUNDING Embeddings make meaning searchable, and RAG uses that search to ground answers Prepare knowledge Chunk approved documents and attach permissions, provenance, and update rules. Create embeddings Encode chunks as vectors so related meanings are located near each other. Retrieve evidence Match the request to relevant, authorized chunks at run time. Generate with citations Give the model retrieved evidence and ask it to answer from that evidence. Source: Lewis et al. — Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks [Sources]
- Lewis et al. — Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — arXiv — https://arxiv.org/abs/2005.11401
AI project onboarding · Shared vocabulary 6 ADAPTATION CHOICE Prompting steers a model now, while fine- tuning changes behavior through training Verdict: Start with prompting and retrieval; fine-tune only when evidence shows a persistent gap. RECOMMENDED Prompting Instructions and examples at request time Fine-tuning Additional training on representative examples Best when The task or policy changes often Behavior needs a stable, repeatable pattern Changes Input context and instructions Learned model behavior Typical project work Design prompts, templates, retrieval Curate, label, split, and test a training set Main governance need Version prompts and test regressions Manage data quality, privacy, and evaluation Source: OpenAI — Prompt engineering ; OpenAI — Fine-tuning guide [Sources]
- OpenAI — Prompt engineering — OpenAI — https://platform.openai.com/docs/guides/prompt-engineering
- OpenAI — Fine-tuning guide — OpenAI — https://platform.openai.com/docs/guides/fine-tuning
AI project onboarding · Shared vocabulary 7 RELIABILITY Hallucinations are plausible but unsupported outputs, so systems need multiple reduction layers Ground each answer in approved evidence Constrain important outputs Clarify or escalate ambiguity Govern the knowledge base Retrieve trusted sources and require citations where users need to verify an answer. Use structured schemas, validation, and policy checks before a response or action is accepted. Ask for missing context and route high- consequence cases to a qualified person. Set provenance, access, freshness, and review rules for the sources used at run time. Source: Lewis et al. — Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks ; OpenAI — Function calling guide ; NIST — AI Risk Management Framework [Sources]
- Lewis et al. — Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — arXiv — https://arxiv.org/abs/2005.11401
- OpenAI — Function calling guide — OpenAI — https://platform.openai.com/docs/guides/function-calling
- NIST — AI Risk Management Framework — NIST — https://www.nist.gov/itl/ai-risk-management-framework
AI project onboarding · Shared vocabulary 8 CONTROLLED ACTION Tool use lets a model act, and function calling supplies the controlled contract 01 Declare allowed tools Describe each operation with a name, purpose, and typed input schema. 02 Request a structured call The model selects a tool and produces arguments that match the schema. 03 Validate and execute The application checks authorization, policy, and inputs before acting. 04 Observe the result Return a bounded result to the model and log the action for review. Source: OpenAI — Function calling guide ; NIST — AI Risk Management Framework [Sources]
- OpenAI — Function calling guide — OpenAI — https://platform.openai.com/docs/guides/function-calling
- NIST — AI Risk Management Framework — NIST — https://www.nist.gov/itl/ai-risk-management-framework
AI project onboarding · Shared vocabulary 9 AI AGENTS Agents repeat plan–act–observe loops, with memory treated as managed state Plan Interpret the goal, choose the next step, and define what success looks like. Act Generate a response or invoke an approved tool within explicit limits. Observe Read the tool result or user feedback and decide whether more work is needed. Manage memory Keep only useful state; separate short-lived context from durable, governed records. Source: Anthropic — Building effective agents ; NIST — AI Risk Management Framework [Sources]
- Anthropic — Building effective agents — Anthropic — https://www.anthropic.com/engineering/building-effective-agents
- NIST — AI Risk Management Framework — NIST — https://www.nist.gov/itl/ai-risk-management-framework
AI project onboarding · Shared vocabulary 10 QUALITY AND SAFETY Evaluation and guardrails make quality, safety, and operational control testable Define success and failure Test before release Guard at run time Monitor in production Build representative tasks, expected outcomes, and unacceptable failure modes into an evaluation set. Compare quality, safety, and tool-use behavior against agreed acceptance criteria. Apply access controls, content policies, schemas, approvals, and human escalation. Log outcomes, analyze failures, and refresh evaluations as the product and knowledge change. Source: OpenAI — Evaluation best practices ; NIST — AI Risk Management Framework [Sources]
- OpenAI — Evaluation best practices — OpenAI — https://platform.openai.com/docs/guides/evals
- NIST — AI Risk Management Framework — NIST — https://www.nist.gov/itl/ai-risk-management-framework
AI project onboarding · Shared vocabulary 11 OPERATING TRADE-OFFS Capability, cost, and latency form a design trade-off to optimize deliberately HIGH CAPABILITY, LOW PRESSURE Use the best fit model; allow deeper reasoning and richer context. HIGH CAPABILITY, HIGH PRESSURE Route selectively, cache reusable work, and keep agent loops bounded. ROUTINE, LOW PRESSURE Prefer simpler models and batch work where the experience permits. ROUTINE, HIGH PRESSURE Use narrow tasks, compact context, and fast deterministic controls. Required capability and judgment High Low Low High Cost and latency sensitivity Set a service-level target first; then choose a model, context strategy, and control loop to meet it. Source: OpenAI — Latency optimization ; OpenAI — Tokens and tokenization [Sources]
- OpenAI — Latency optimization — OpenAI — https://platform.openai.com/docs/guides/latency-optimization
- OpenAI — Tokens and tokenization — OpenAI — https://platform.openai.com/docs/concepts/tokens
AI project onboarding · Shared vocabulary 12 FROM VOCABULARY TO CHARTER A short discovery sequence turns the vocabulary into an AI project charter Next step Owner Due Select a narrow user problem and define the decision or outcome it should improve Business sponsor Discovery start Specify the context, trusted knowledge, tools, and permissions the system may use Product and engineering Solution design Define evaluation cases, failure thresholds, guardrails, and escalation paths Product and risk owner Before pilot Set quality, cost, and latency targets before choosing the production architecture Technology lead Before build Source: OpenAI — Evaluation best practices ; NIST — AI Risk Management Framework ; OpenAI — Latency optimization [Sources]
- OpenAI — Evaluation best practices — OpenAI — https://platform.openai.com/docs/guides/evals
- NIST — AI Risk Management Framework — NIST — https://www.nist.gov/itl/ai-risk-management-framework
- OpenAI — Latency optimization — OpenAI — https://platform.openai.com/docs/guides/latency-optimization