9 Types of AI Models for Developers Building With LLMs (2026)

OpenAI's model catalog now splits into separate lists for text, image, transcription, speech, and realtime models, and Google's Gemini catalog is organized the same way. Picking "the model" for an app has quietly turned into picking several.

That matters because each type is built for a different job. A reasoning model will answer a classification question, but it costs more and runs slower than a small model that answers it just as well, and an LLM asked to search your documents still needs an embedding model to find them.

Most explanations of AI model types sort them by how they learn: supervised, unsupervised, reinforcement. That's accurate, but it doesn't help when you're deciding what to call from your code. Sorting by the job each model does is more useful for a developer.

These are the nine types of AI models worth knowing if you're building with LLMs, what each one is for, and how to choose and combine them in one app.

What Is an AI Model?

An AI model is a program that has learned a pattern from data and applies it to new inputs. Give it an input, such as a prompt, an image, or a row of customer data, and it returns an output, such as text, a label, a number, or another image.

What the model learned lives in its parameters, also called weights: the numbers adjusted during training until the model's outputs matched the training data closely enough. A frontier LLM has billions of them, while a fraud-detection model trained on transaction data might have a few thousand. In both cases, the model is the trained artifact, and the code that calls it is separate.

How Do AI Models Work?

Every AI model goes through the same phases, and most developers building with LLMs only work with the last one.

1. Training - The model sees examples, makes predictions, measures how wrong they were, and adjusts its weights to be less wrong next time. Training a frontier model takes compute budgets that only a handful of labs have.
2. Fine-tuning (optional) - A trained model keeps training on your own data so it handles a narrower task better, without starting from scratch.
3. Inference - The weights are fixed, and the model turns new inputs into outputs. This is what runs when you call a model through an API or run open weights on your own hardware.

For LLMs, inference has a few moving parts that shape how you build with them:

• Tokens - The model breaks text into tokens, which are chunks of words, and API pricing is usually per token in and out.
• Next-token prediction - The model predicts the most likely next token given everything before it, then repeats until the response is done.
• Attention - The transformer architecture behind modern LLMs weighs which earlier tokens matter for each prediction.
• Context window - This caps how many tokens the model can consider at once, counting your prompt, the conversation history, and the response.

In practice, those pieces explain most of what you'll tune: token counts drive cost and latency, and the context window decides how much history and retrieved content fits in a single call.

9 Types of AI Models

Each type below is defined by the job it does in an application. Some overlap, since a multimodal model is usually also an LLM, but each one maps to a decision you'll make when building.

Large Language Models (LLMs)

Large language models are transformer-based models trained on huge amounts of text to predict the next token, which turns out to be enough to write, summarize, translate, answer questions, write code, and call tools. They're the default starting point for most AI features, and the model at the center of almost every agent.

The main options come from OpenAI's GPT models (GPT-6 Astra and GPT-6.1 Sol), Anthropic's Claude (Claude Opus 5.5), and Google's Gemini (Gemini 3.8 Flash). The largest and most capable of these are usually called frontier AI models, and they're also the most expensive per token. As a result, using one for every request in an app is one of the more common ways LLM bills grow faster than traffic.

Reasoning Models

Reasoning models spend extra tokens working through a problem before they answer, which makes them stronger on multi-step math, complex code changes, and agent planning. Google lists Gemini 4 Argon as its flagship reasoning model, OpenAI's GPT-6 models expose a reasoning effort setting, and DeepSeek turns on its thinking mode with a request parameter.

What you give up is speed and cost. Those thinking tokens are billed and take time to generate, so a reasoning model on a simple lookup pays for work the task didn't need. If you're weighing options, the best reasoning models differ a lot in what they charge for that thinking.

Small Language Models (SLMs)

Small language models have parameter counts low enough to run on a laptop, a phone, or a single GPU. Google's Gemma 4 comes in E2B and E4B sizes, small enough for that kind of hardware, and because the weights are open, you can run them on your own hardware with no per-token API bill.

SLMs handle narrow, high-volume tasks well: extracting fields, short classifications, rewriting, and simple routing decisions. They fall behind larger models on open-ended reasoning and long, complex instructions, so they tend to work best as one step in a larger pipeline. The tradeoff between running an open model yourself and calling a hosted one is laid out in Gemma vs. Gemini, which compares both from the same vendor.

Multimodal Models

Multimodal models accept or produce more than one kind of data, such as text, images, audio, or video. OpenAI's GPT-6 models take text and image input, and Google's Gemini Omni Flash generates and edits video.

For developers, multimodal input is the part that changes what an app can do. A support agent can read a screenshot of an error, an expense tool can pull line items from a photo of a receipt, and a document pipeline can parse a scanned PDF without a separate OCR step. Images cost tokens like text does, though, so a pipeline that sends full-resolution screenshots on every call may cost more than you expect.

Embedding Models

Embedding models turn text, and sometimes images, into vectors: lists of numbers where similar meanings sit close together. They don't generate anything a user reads. Instead, they power search, retrieval-augmented generation (RAG), recommendations, clustering, and duplicate detection.

Google offers gemini-embedding-001 through the Gemini API and EmbeddingGemma 2 as an open model. Picking from the best embedding models matters because the one you choose decides which documents your RAG pipeline retrieves, and a weak one shows up as an LLM answering confidently from the wrong context. Because of that, switching embedding models means re-embedding everything already stored in your vector database.

Image Generation Models

Image generation models create or edit images from a text prompt, a reference image, or both. Current options include OpenAI's GPT-Image-2.5, Google's Nano Banana 2.1, and Black Forest Labs' FLUX 3.

Many image models are built on diffusion, a technique that starts from random noise and removes it step by step until an image matching the prompt appears. In an app, these models handle product mockups, marketing assets, UI placeholders, and image edits. Pricing models vary by vendor, so check whether a model bills per image or per token before estimating costs.

Speech Models

Speech models convert between audio and text. Speech-to-text models, such as OpenAI's gpt-transcribe and Google's Gemini 3.5 Transcribe, turn recordings into transcripts. Text-to-speech models, such as OpenAI's gpt-4o-mini-tts and Gemini 3.8 Flash TTS, turn text into spoken audio.

A third group, realtime models like OpenAI's GPT-Realtime-2.1 and Gemini 3.8 Live, handle a voice conversation end to end. Voice agents usually face a choice between that single realtime model and a chain of transcription, an LLM, and text-to-speech. The chain gives you more control over each step, while the realtime model usually responds faster.

Classification Models

Classification models assign a label or a score to an input: safe or unsafe, spam or not, which intent, which category. In LLM apps, they most often show up as guardrails that check a prompt before it reaches the main model, or as evaluators that check an output after.

Meta's Llama Guard 4 (12B) classifies prompts and responses against safety categories, and Google's ShieldGemma 2 is another safety classifier with open weights. Span-01, Respan's classifier for judging traces, takes a behavior definition written in plain language and returns present, absent, or not_observable probabilities for each one, checking several behaviors against a trace in one pass. Span-02, Respan's open-weight decision model, answers yes/no, multiple-choice, or numeric questions with probabilities over the choices, and you can cap how long it thinks.

Traditional Machine Learning Models

Traditional machine learning models, such as logistic regression, random forests, and gradient-boosted trees like XGBoost, predict outcomes from structured data. Fraud scores, churn risk, demand forecasts, and ranking signals commonly run on them.

They're cheap to run, fast enough for per-request decisions, and easier to explain than a neural network. When your input is a table of numbers rather than language, a traditional model can beat an LLM on cost and hold up on accuracy. LLM apps still use them for supporting jobs, like scoring which requests need a human review.

How to Choose an AI Model Type

The quickest way to narrow the choice is to start from the task's input and output. Text in and text out points to an LLM, an SLM, or a reasoning model. Documents and images point to a multimodal model, audio points to a speech model, and anything involving search points to an embedding model alongside one of the others. From there, three questions usually settle it.

Open-Weight vs. Proprietary

Open-weight models, such as Gemma 4 and DeepSeek's models, publish their weights, so you can download and run them on your own infrastructure. That gives you control over where data goes and removes the per-token bill, but you take on the hosting, scaling, and GPU costs, and license terms vary by model.

Proprietary models from OpenAI, Anthropic, and Google run only through the vendor's API. They're simpler to start with and include most of the frontier models, while the tradeoff is less control over pricing changes and deprecations. If self-hosting is the deciding factor, the field of open-source LLMs now covers a wide range of text tasks.

Cost and Latency

Within a type, larger models cost more per token and take longer to respond, and reasoning models add billed thinking tokens on top. A reasoning model can take seconds on a request that a small model answers in a fraction of that time.

Because of that, matching the model to the step tends to save more than negotiating a lower rate. An agent that sends simple tool-call formatting to a small model and saves the frontier model for planning can cut its bill noticeably without losing quality where it counts. Caching and prompt trimming help too, and LLM cost optimization covers the full set of levers.

Testing on Your Own Data

Public LLM benchmarks measure how a model does on someone else's tasks. They don't tell you how it handles your prompts, your users' phrasing, or your edge cases, which is where a model that ranks well can still fall short.

Before committing to a model type, build a small test set from real inputs your app sees, run each candidate against it, and score the outputs with the same criteria you'd use in production. It's slower than reading a leaderboard, but it's the closest you'll get to predicting how the model behaves for your users. For a full walkthrough of building that test set, see how to test AI models.

Combining AI Model Types in One App

Production AI apps rarely run on a single model type. A few combinations come up again and again:

• RAG - An embedding model finds the relevant documents, and an LLM answers from them.
• Voice agents - A speech-to-text model transcribes the user, an LLM decides what to say, and a text-to-speech model speaks the reply.
• Guardrails - A classification model checks each prompt before it reaches the main LLM, so unsafe requests are blocked before they cost anything.
• Agents - A reasoning model plans the steps, a frontier LLM writes the final response, and a small model handles the repetitive formatting in between.

Each choice makes sense on its own, while together they add up to several providers, several API keys, and several places a request can fail.

That's the problem an LLM gateway exists to solve. It puts every model behind one API, so switching a step to a different model is a configuration change rather than a code change, and every call lands in one place where you can see what it cost and whether it worked.

Route Each Task to the Right Model With Respan

Once an app mixes model types, someone has to maintain the routing: a separate SDK and key for each provider, fallback logic for each one, and spend that's hard to trace back to the step that caused it. Respan is the AI router with built-in observability and automated evals. Through the Respan AI gateway, one API reaches 1,000+ models across every major provider, and switching models means changing a single word in your request.

Respan Router (beta) goes a step further and picks the model for each task. It reads the system prompt and the latest exchange to classify the task, checks each candidate against the request's context length and needs like images or tools, and keeps a conversation on its current model while that model still fits. You pay the serving model's rates, including cache discounts, with no markup.

• Stay up when a model fails - List your fallback models once, and Respan moves to the next one when a provider errors or rate-limits.
• See every step in one trace - Each LLM call, tool run, and retrieval becomes a span with its own input, output, latency, and cost, so a bad answer traces back to the step that produced it.
• Know where every dollar goes - Costs break down by model, by request, and by end customer, with soft and hard spend limits per key or per customer.
• Catch regressions before users do - Evals score output on a test set before you ship a model change, then keep scoring live production traffic after.
• Serve repeat requests from cache - An exact repeat request comes back from cache near-free and instantly.
• Run the gateway yourself - Respan's open-source gateway, currently pre-alpha, runs on your own infrastructure and puts chat, embeddings, image generation, transcription, speech, and video calls behind one gateway. Turn on its Respan plugin when you want every request traced, priced, and scored in the hosted platform.

Instead of reading logs after the fact, use Respan to run observability in production, know when production shifts, and act before it spreads.

Switch Models With One Line of Code

If your app already uses the OpenAI SDK, moving it onto Respan means changing the base URL and the API key. From there, switching models is a change to the model string, and setting it to span-router hands the choice to Respan Router.

import os
from openai import OpenAI
 
client = OpenAI(
    base_url="https://api.respan.ai/api/",
    api_key=os.environ["RESPAN_API_KEY"],
)
 
# Call a specific model
response = client.chat.completions.create(
    model="openai/gpt-6-astra",
    messages=[{"role": "user", "content": "Summarize this support ticket."}],
)
 
# Or let Respan Router pick the model for the task
response = client.chat.completions.create(
    model="span-router",
    messages=[{"role": "user", "content": "Summarize this support ticket."}],
    extra_body={"thread_identifier": "conv-8f2c"},
)

Both calls are logged as spans automatically, with latency and cost attached, so the trace is there from the first request.

FAQ

What are different types of AI models called?

The types developers work with most are large language models (LLMs), reasoning models, small language models (SLMs), multimodal models, embedding models, image generation models, speech models, classification models, and traditional machine learning models. AI models are also grouped by how they learn, such as supervised, unsupervised, and reinforcement learning, which describes training rather than what the model does in an app.

How many types of AI are there?

It depends on the framework. By capability, AI is usually split into three types: narrow AI, general AI, and superintelligent AI, and only narrow AI exists today. By functionality, it's often split into four: reactive machines, limited memory, theory of mind, and self-aware AI. Every model a developer can call today, including LLMs, is narrow AI.

Is ChatGPT an LLM or generative AI?

Both. ChatGPT is an application built on OpenAI's GPT models, which are large language models, and LLMs are one kind of generative AI because they generate new content. Current GPT models are also multimodal, since they accept image input alongside text.

What are the different branches of artificial intelligence?

The main branches are machine learning, deep learning, natural language processing, computer vision, robotics, and expert systems. Deep learning is a subset of machine learning, and modern natural language processing and computer vision both run mostly on deep learning models like transformers.

What are the two main types of generative AI models?

The pairing cited most often is generative adversarial networks (GANs) and transformer-based models. GANs were among the most widely used generative techniques until transformers took over text generation. Variational autoencoders (VAEs) and diffusion models are also common, and diffusion now drives many image generation models.