AI Words, Explained

A model doesn't read letters or words. Text gets chopped into tokens — chunks of roughly ¾ of an English word. "Unbelievable" might be un · believ · able. Everything about a model is measured in tokens: pricing, speed, memory limits. It's the kilogram of the AI world.

"The quick brown fox" → ["The", " quick", " brown", " fox"]   // 4 tokens
"unbelievable"        → ["un", "believ", "able"]            // 3 tokens
"你好世界"             → often 1 token per character or less

// rule of thumb: 1,000 tokens ≈ 750 English words
// API bills you per token, in AND out

The prompt is everything you send the model — not just your question, but the instructions, examples, and documents around it. The system prompt is a special first message the user never sees: the model's job description ("You are a support agent for Acme; never discuss competitors"). When an app's AI behaves with a personality, that's the system prompt talking.

// what actually gets sent to the API:
{
  "system": "You are a terse code reviewer. Point at lines. No praise.",
  "messages": [
    { "role": "user", "content": "Review this function: ..." }
  ]
}

The context window is how many tokens the model can look at at once — its desk size. A "200K context" model can spread ~150,000 words on the desk. Crucially: the model has no memory between conversations. Each new chat starts with an empty desk, and "remembering" mid-conversation means the app quietly re-sends the whole history every single turn.

// what "the AI remembers our chat" actually is:
turn 1:  send [msg1]                    → answer1
turn 2:  send [msg1, answer1, msg2]     → answer2
turn 3:  send [msg1, answer1, msg2, answer2, msg3] → ...

// nothing is stored in the model. The app re-mails the
// entire correspondence with every new letter.

A hallucination is the model stating something false with total confidence — a court case that doesn't exist, an API function that was never written. It's not lying (lying needs intent). The model is a next-word predictor: when the true answer is missing from its knowledge, the most statistically plausible-sounding answer comes out instead, in perfect grammar.

// the classic failure shape:
Q: "What does the pandas function df.smooth_outliers() do?"
A: "df.smooth_outliers() applies rolling-window winsorization
   to columns exceeding 3 standard deviations..."   // ← does not exist.
                                                    // described beautifully.

At each step the model has a ranked list of plausible next tokens. Temperature decides how it picks: at 0, always take the top choice (same answer every time, nearly); higher values give lower-ranked tokens a chance (varied, creative, riskier). It's not an "intelligence" dial — it's a dice dial.

next token after "The capital of France is":
  " Paris"  92%   ← temperature 0 takes this, always
  " the"     3%
  " located" 2%   ← temperature 1.0 might wander here
  ...

// extraction, code, grading  → temperature 0
// brainstorming, naming, fiction → 0.7 - 1.0

An embedding turns text into a list of numbers — coordinates in a space where similar meanings land close together. "How do I reset my password" and "forgot login credentials" share almost no words, but their embeddings are neighbors. That's what powers "search by meaning" — and it's the engine inside RAG, next section.

embed("How do I reset my password?")  → [0.12, -0.98, 0.44, ...]  // ~1000 numbers
embed("forgot my login credentials")  → [0.11, -0.95, 0.47, ...]  // ← nearby!
embed("best pizza in Naples")         → [-0.71, 0.22, -0.30, ...] // ← far away

// a "vector database" is just a tool for finding
// nearest neighbors in this space, fast.

Retrieval-Augmented Generation: instead of asking the model to answer from memory (closed-book, hallucination-prone), the app first searches your documents for relevant passages — usually with embeddings — and pastes them into the prompt. The model answers from what's on the desk. Every "chat with your PDF / docs / knowledge base" product is this.

user asks: "What's our refund policy for annual plans?"

1. embed the question, find nearest chunks in your docs
2. build the prompt:
   "Using ONLY these excerpts:
    [chunk 12: 'Annual plans may be refunded within 30 days...']
    [chunk 47: 'Refunds are prorated after...']
    Answer: What's our refund policy for annual plans?"
3. model answers from the excerpts — and can cite them

Fine-tuning continues a model's training on your own examples, changing its default behavior: tone, format, style, domain vocabulary. The crucial distinction — fine-tuning teaches skills and habits, RAG provides facts. Mixing these up is the most expensive misunderstanding in applied AI.

// want the model to KNOW your product docs?     → RAG
// want it to WRITE in your support team's voice? → fine-tune
// want both? → both. they're not competitors.

✗ "We fine-tuned on our wiki so it knows our policies"
   // it now SOUNDS like your wiki. facts still fuzzy,
   // and last week's policy change? not in there.
✓ fine-tune the tone, RAG the knowledge

A chatbot answers once. An agent runs in a loop: look at the goal → choose a tool (search, run code, read a file, call an API) → observe the result → decide the next step → repeat until done. The "tools" are functions the developer hands the model, with descriptions. Agency isn't a different kind of model — it's a model given hands and permission to iterate.

goal: "Find why the deploy failed and fix it"

loop 1: tool=read_logs        → sees: ImportError in app.py
loop 2: tool=read_file(app.py) → sees the broken import
loop 3: tool=edit_file         → fixes the line
loop 4: tool=run_tests         → green ✓ → report & stop

MCP (Model Context Protocol) is a standard plug for connecting AI apps to tools and data. Before: every app wrote custom code for every tool (Slack, GitHub, your database) — M×N integrations. With MCP, a tool exposes one MCP server, any AI app that speaks MCP can use it. Think USB: one port, everything connects. A skill, meanwhile, is a packaged playbook — instructions (often a folder with a markdown file and scripts) that teach the model how to do a specific task your way.

// MCP = access to things:
your AI app ──MCP──> github server    (read PRs, file issues)
            ──MCP──> postgres server  (query the database)
            ──MCP──> slack server     (read/send messages)

// skill = knowing how to do a task:
skills/deploy-checklist/SKILL.md
  "Before deploying: run tests, check migrations,
   post in #deploys, then..."

// tool gives hands. skill gives the recipe.

The whole page in eleven lines. Steal for your next meeting.

token           the model's syllable; everything is priced in these
prompt          everything you send: question + instructions + examples
system prompt   the hidden job description
context window  the desk, not the memory — chats re-send everything
hallucination   confident autocomplete filling a knowledge gap
temperature     the dice dial: 0 = same answer, 1 = adventurous
embedding       meaning as coordinates; neighbors = similar
RAG             open-book exam: search your docs, paste, then answer
fine-tuning     teaches habits & tone — NOT facts (that's RAG)
agent           a model in a loop with tools and a goal
MCP             the USB port: one standard plug for AI ↔ tools
skill           a packaged playbook: how to do one task, your way

Vocabulary is leverage: half of AI confusion is people using the same word for different things. You now hold the working definitions — the next time someone says "we'll just fine-tune it on our docs," you'll know exactly which question to ask.