Two websites both say they use AI. One answers questions about an uploaded handbook. The other generates text from a blank chat. They might even use the same underlying model. Why do they behave differently?
Before you begin: Know the difference between a model and a learning task.
Models, providers, applications, and workflows
The model is the learned computation that produces outputs from inputs. The runtime or provider makes that model available for use. The application adds instructions, data, tools, storage, and an interface. The workflow is how a person or organization uses that application to complete a job.
For a workshop help desk, the model may generate a reply. The application retrieves the current timetable and limits which documents are available. The workflow requires an organizer to approve changes to registration. If the reply contains last month's timetable, the problem might be stale retrieval data rather than the model's general language ability.
When you compare two AI products, ask which of these parts differs. A different interface can expose tools or documents that the same model does not have in another application.
Text, image, and audio models
A text model accepts and produces text representations. A multimodal model can work with more than one kind of input or output, such as images, audio, or text. “Multimodal” does not tell you which combinations a specific product supports. A product that reads an image may not generate one, and a model that handles audio may be exposed through an application that accepts only text.
An assistant can also call a separate tool. If it searches the web, runs code, or reads a file, those actions may involve systems outside the language model. Ask what information the tool returned and how the application used it. The model's answer and the tool's evidence are related but distinct.
Which part does your application need?
A model name can hide several decisions. A user sees an application. The application sends requests through an interface, perhaps retrieves documents or runs tools, and eventually invokes a model. If the answer is wrong, the useful repair depends on which layer lost the needed information or failed to check the result.
Treat the comparison as a way to ask a more precise question at a meeting. Replacing a model might improve language generation, but it does not automatically repair a broken document index or a missing sign-in check. When reading older product announcements, separate the lasting architectural idea from the availability and pricing that may have changed.
A member asks an assistant about the current timetable.
Replace the language model while leaving the application’s stale timetable source untouched.
The system still supplies stale evidence. A different model has not updated that source.
Hosted APIs and downloadable models
With a hosted API, you send requests to a service. The provider operates the model infrastructure, while you remain responsible for the data you send and your application's behavior. With a model you run yourself, you manage the hardware and runtime. You still need to inspect its license, requirements, and limitations.
“Open weights” means model parameters are available under specified terms. It does not by itself mean unrestricted use, a complete public training dataset, or an entirely reproducible training process. “Local” means computation happens on a chosen machine; it does not prove that the whole application is offline or that its logs stay private.
Choose a setup for a document helper
You need a helper for public workshop questions. It must quote the current cancellation policy and refuse to guess when the policy is missing. Start with three checks: can you provide the policy, can the answer show where its statement came from, and can you test a missing-answer case?
Next, check cost, response time, accessibility, and data handling. A model with a larger benchmark score may still be a worse product choice if the application cannot attach the right evidence. A small retrieval-and-answer system may fit the job better than an autonomous agent with broad tools.
Write down the exact product, model identifier if exposed, access plan, and review date when you compare options. Availability, limits, and terms can change independently. This course avoids presenting a permanent winner where the answer depends on the task and account.
Check an AI product claim
A tool advertises “private AI with the latest knowledge.” What two questions would you ask before uploading a document?
Turn the claim into checkable questions
For privacy, ask where processing occurs, what is retained, who can access it, and whether your account's controls meet your needs. For current knowledge, ask whether answers use live retrieval, how source dates are exposed, and what happens when retrieval fails. A broad slogan does not answer either question. You can test freshness with a public document that changed recently without first uploading private material.
The next lesson looks inside the language model itself. Keep this layered map in mind: knowing how a model works will not explain every behavior of an application built around it.
Practice with feedback
Separate the four layers of an AI product
A colleague says "we should just use GPT for this". You need to work out what part of the stack they are actually proposing.
Place model, access, application, and user interface correctly, then justify one access decision.
Check your understanding
Your task
Write a one-page decision record for a handbook assistant, choosing an access route with reasons you could defend in three months.
These notes stay on this page. Download them before leaving.
What to include
- Input and output are concrete artefacts, not goals
- All four layers are filled with distinct things
- The choice cites at least one measurable factor such as monthly request volume or a data boundary
- The revisit trigger is observable, for example "if volume passes 200k requests a month"
Compare with a worked answer
Here is one way to answer. Check how it uses the information in the task.
Feature: answer staff questions about the employee handbook, with the source paragraph shown. Input: a question in plain English plus the asker's department. Output: an answer of at most 120 words, with the handbook section number, or the sentence "the handbook does not cover this". How I check the output: 40 written questions with known correct sections. I count how often the cited section is right, separately from how often the prose is right.
Layer map Model: a general instruction-following model; I am not training one. Access: a hosted API over HTTPS with a per-token price. Application: retrieval over handbook sections, prompt assembly, a check that the cited section exists, and logging. Interface: a search box inside the existing intranet page.
Access options I considered: A) Hosted API. No hardware, pay per request, handbook text leaves our network under a data processing agreement. B) Open-weight model on our own GPU. Fixed monthly hardware cost, text never leaves, we own upgrades and outages.
Choice and the two reasons that decided it: Hosted API. First, expected volume is about 3,000 questions a month, which is far below the point where a dedicated GPU pays for itself. Second, the handbook is already shared with an external payroll vendor, so this adds no new category of data exposure.
What would make me revisit this: Volume above roughly 150,000 requests a month, or a legal requirement that handbook content stay on our own network. Either one flips the calculation, and I have written the numbers down so the next person can redo it.
When you are signed in, opening the challenge carries your edited working notes into its draft in this browser. The challenge has its own completion record. Practising here does not award points or mark it complete.
Further reading
The original Model Cards for Model Reporting proposes documenting intended uses, evaluations, and limitations. The Hugging Face model-card documentation shows how these records are represented alongside models.