Question 1

What is Llama 3.1 8B Instruct Turbo?

Accepted Answer

Llama 3.1 8B Instruct Turbo is a model from Meta, released on Jul 23, 2024. Llama 3.1 8B Instruct Turbo is an open-weight Llama-class model commonly used for chat, fine-tuning, and on-prem or hosted inference. Llama 3.1 8B Instruct Turbo is an open-weight Llama-class model commonly used for chat, fine-tuning, and on-prem or hosted inference. Supports tool use. Up to 131K context window.

Question 2

How much does Llama 3.1 8B Instruct Turbo cost?

Accepted Answer

On the Respan gateway, input starts at $0.02 per 1M tokens on DeepInfra, and output starts at $0.04 per 1M tokens on DeepInfra, after gateway discounts. Averaged across 2 providers, it costs $0.10 per 1M input tokens and $0.11 per 1M output tokens.

Question 3

Which providers serve Llama 3.1 8B Instruct Turbo on Respan?

Accepted Answer

2 providers serve Llama 3.1 8B Instruct Turbo: DeepInfra and TogetherAI. TogetherAI requires your own provider API key (BYOK).

Question 4

What is Llama 3.1 8B Instruct Turbo's context window?

Accepted Answer

Llama 3.1 8B Instruct Turbo accepts up to 131K tokens of context on its largest provider route.

Question 5

How do I use Llama 3.1 8B Instruct Turbo with the Respan gateway?

Accepted Answer

Point any OpenAI-compatible SDK at https://api.respan.ai/api/ with your Respan API key and set the model to "llama-3.1-8b-instruct-turbo". To set it up with a coding agent, copy the agent setup prompt at the top of this page.

Question 6

How fast is Llama 3.1 8B Instruct Turbo?

Accepted Answer

Over the last 30 days of requests through the Respan gateway, the fastest provider was DeepInfra at 13 tokens per second, and the lowest response latency was 0.6 seconds on DeepInfra. Speed varies with request size and provider.

Provider	Context	Input	Output	Capabilities
DeepInfra`deepinfra/meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo`	131K	$0.02/M	$0.04/M
TogetherAIBYOK only`together_ai/meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo`	131K	$0.18/M	$0.18/M

Llama 3.1 8B Instruct Turbo

Providers

Certified where it counts.

Pricing

Cost by volume

Throughput and latency

Output speed

Response latency

Apps