Llama 3 8B Instruct
Meta's Llama 3 model is rolled out in varying sizes, with the 8B instruct-tuned variant particularly optimized for high-quality dialogue purposes. The performance of Llama 3 has been found to be strong when compared to leading closed-source models, backed by human evaluations. The model family includes 8B and 70B sizes, with pretrained and instruct-tuned variants suitable for assistant-like chat. Llama 3 leverages auto-regressive language modeling with optimized transformer architecture. Training data is a mix of publicly available online data, with a knowledge cutoff in March 2023 for 8B and December 2023 for 70B models. The models use Grouped-Query Attention for improved inference scalability.
- Tool Use
main.py
from openai import OpenAI client = OpenAI( base_url="https://api.respan.ai/api/", api_key="YOUR_RESPAN_API_KEY",) response = client.chat.completions.create( model="novita/meta-llama/llama-3-8b-instruct", messages=[{"role": "user", "content": "Hello!"}], extra_body={ "fallback_models": ["meta-llama/llama-3-8b-instruct"], },)print(response.choices[0].message.content)from openai import OpenAI client = OpenAI( base_url="https://api.respan.ai/api/", api_key="YOUR_RESPAN_API_KEY",) response = client.chat.completions.create( model="novita/meta-llama/llama-3-8b-instruct", messages=[{"role": "user", "content": "Hello!"}], extra_body={ "fallback_models": ["meta-llama/llama-3-8b-instruct"], },)print(response.choices[0].message.content)Providers
Gateway routes for this model on the Respan gateway, including fallback routes when configured.
Similar
Other models from the same provider available through the gateway.
Certified where it counts.
Built to meet the security and privacy standards that enterprise and healthcare teams require.
ISO 27001
The internationally recognized standard for information security management.
SOC 2
Secure, compliant management of your data across all of our systems.
GDPR
Operated under GDPR, the world's strictest standard for data privacy.
HIPAA
HIPAA compliant, with a BAA available for healthcare teams.
Works with your entire stack
Use Respan with your favorite frameworks and tools.
