Question 1

What is Llama 3 8B Instruct?

Accepted Answer

Llama 3 8B Instruct is a model from Meta, released on Apr 18, 2024. Meta's Llama 3 model is rolled out in varying sizes, with the 8B instruct-tuned variant particularly optimized for high-quality dialogue purposes. The performance of Llama 3 has been found to be strong when compared to leading closed-source models, backed by human evaluations.

The model family includes 8B and 70B sizes, with pretrained and instruct-tuned variants suitable for assistant-like chat. Llama 3 leverages auto-regressive language modeling with optimized transformer architecture.

Training data is a mix of publicly available online data, with a knowledge cutoff in March 2023 for 8B and December 2023 for 70B models. The models use Grouped-Query Attention for improved inference scalability.

Question 2

How much does Llama 3 8B Instruct cost?

Accepted Answer

On the Respan gateway, input starts at $0.20 per 1M tokens on TogetherAI, and output starts at $0.20 per 1M tokens on TogetherAI, after gateway discounts.

Question 3

Which providers serve Llama 3 8B Instruct on Respan?

Accepted Answer

2 providers serve Llama 3 8B Instruct: TogetherAI. TogetherAI require your own provider API key (BYOK).

Question 4

What is Llama 3 8B Instruct's context window?

Accepted Answer

Llama 3 8B Instruct accepts up to 8K tokens of context on its largest provider route.

Question 5

How do I use Llama 3 8B Instruct with the Respan gateway?

Accepted Answer

Point any OpenAI-compatible SDK at https://api.respan.ai/api/ with your Respan API key and set the model to "llama-3-8b-instruct". To set it up with a coding agent, copy the agent setup prompt at the top of this page.

Provider	Context	Input	Output	Capabilities
TogetherAIBYOK only`together_ai/meta-llama/Llama-3-8b-chat-hf`	8K	$0.20/M	$0.20/M	—
TogetherAIBYOK only`together_ai/meta-llama/Meta-Llama-3-8B-Instruct`	8K	$0.20/M	$0.20/M

Llama 3 8B Instruct

Providers

Certified where it counts.

Pricing

Cost by volume

Apps