Together AI: Inference & Compute

Founded 2022|San Francisco, CA|51-200 people|Usage-based

What is Together AI?

Together AI is a cloud-based platform for building with open-source generative AI, founded on June 11, 2022 in San Francisco by Ce Zhang, Chris Re, Percy Liang, and Vipul Ved Prakash. The company raised USD 305 million in Series B funding in 2025 with participation from industry leaders including NVIDIA and Salesforce Ventures. Together AI provides serverless inference with pay-as-you-go pricing starting from USD 0.10 per million tokens for small models and USD 0.90 for Llama 3 70B, with a free USD 5 credit to start. The platform offers a 50 percent discount on batch inference and 50 percent savings on prompt caching for repetitive queries. For teams requiring dedicated resources, Together AI provides GPU endpoints billed per minute, with high-end H100 and H200 GPUs available. The platform specializes in open-source model deployment and provides instant GPU clusters for training and inference workloads. Together AI has become a leading platform for teams building with open-source AI models, offering both serverless convenience and dedicated infrastructure options.

Key features

Core capabilities this platform advertises.

Inference and training cloud
Open-source model hosting
Serverless inference endpoints
Fine-tuning as a service
Competitive GPU pricing

Strengths and tradeoffs

What this tool does well, and the limitations to keep in mind.

Pros

Competitive pricing starting at USD 0.10 per million tokens
50 percent discounts on batch inference and prompt caching
Strong backing from NVIDIA and Salesforce Ventures
Flexible deployment with both serverless and dedicated options

Cons

Focused on open-source models may limit access to proprietary models
Per-minute billing for dedicated GPUs can be complex to predict
Newer platform with less extensive documentation than established providers
Pricing varies significantly by model making cost planning difficult

Plans & pricing

What's included in each plan, and how the tiers compare.

Serverless

Starting at $0.10

Per 1M tokens

Pay-as-you-go
Multiple open-source models
$5 free credit
Llama 3 70B at $0.90/M tokens

Batch Inference

50% off serverless

Bulk processing discount
Async execution
Cost optimization

Dedicated GPU

Per minute billing

H100, H200 GPUs
Reserved capacity
Production workloads
Custom pricing

View official pricing page

Common use cases

Developers and companies deploying open-source AI models in production

Hosting open-source LLMs in production
Fine-tuning models on custom data
Cost-efficient inference at scale
Training custom models
Rapid model prototyping

Using Together AI with Respan

Together AI and Respan enable open-source AI deployment with cost tracking. Run models on Together AI while monitoring usage with Respan.

Track costs across serverless and dedicated GPU endpoints
Monitor batch inference efficiency and caching effectiveness
Compare Together AI performance against other inference providers
Optimize model selection based on Respan cost and latency analytics

Run Together AI in production with Respan

10KFree traces/mo

1,000+Models

5 minSetup

Respan lets you trace LLM and agent calls across any model or framework, A/B test prompts on production traffic, and route requests across 1,000+ models through one gateway.

Try Respan free

Best Together AI alternatives & competitors

Top companies in Inference & Compute you can use instead of Together AI.

NVIDIA

H100 and B200 GPU clusters