Model Type | |
Use Cases |
Areas: | |
Applications: | Assistant-like chat, Natural language generation tasks |
|
Primary Use Cases: | Dialogue use cases, Text generation |
|
Limitations: | Use other than English, Violation of laws, Prohibited uses in Acceptable Use Policy |
|
Considerations: | Fine-tuning for languages beyond English allowed under license compliance |
|
|
Additional Notes | Included benchmarks demonstrate performance scaling with model size and modifications to improve model safety. |
|
Supported Languages | |
Training Details |
Data Sources: | Publicly available online data |
|
Data Volume: | |
Methodology: | Supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF) |
|
Context Length: | |
Hardware Used: | Meta's Research SuperCluster and production clusters, third-party cloud compute |
|
Model Architecture: | An auto-regressive language model that uses an optimized transformer architecture |
|
|
Safety Evaluation |
Methodologies: | Extensive red teaming exercises, Adversarial evaluations, CyberSecEval suite, Child Safety risk assessments |
|
Findings: | Llama 3 significantly less likely to falsely refuse prompts than Llama 2, Drastically reduced residual risks while maintaining helpfulness |
|
Risk Categories: | Child Safety, Cyber Security, CBRNE, General Safety |
|
Ethical Considerations: | Promoting safe, fair, and responsible use; transparency and safety measures implemented |
|
|
Responsible Ai Considerations |
Fairness: | Reflects a wide range of backgrounds and perspectives |
|
Transparency: | Open consortium involvement, transparency measures in safety evaluation |
|
Accountability: | Developers and Meta are accountable for ensuring proper use |
|
Mitigation Strategies: | Implemented safety best practices; extensive safety evaluations |
|
|
Input Output |
Input Format: | |
Accepted Modalities: | |
Output Format: | |
Performance Tips: | Use pre-trained and instruction tuned variants appropriately for tasks |
|
|
Release Notes |
Date: | |
Notes: | Initial release of Meta Llama 3 models in 8B and 70B sizes. |
|
|
|