Model Type | text generation, multilingual |
|
Use Cases |
Areas: | |
Applications: | Assistant-like chat, Natural language generation tasks |
|
Primary Use Cases: | Multilingual dialogue, Synthetic data generation |
|
Limitations: | Usage in unsanctioned languages or illegal activities prohibited. |
|
Considerations: | Developers responsible for additional finetuning for unsupported languages. |
|
|
Supported Languages | English (High), German (High), French (High), Italian (High), Portuguese (High), Hindi (High), Spanish (High), Thai (High) |
|
Training Details |
Data Sources: | Publicly available online data |
|
Data Volume: | |
Methodology: | Supervised fine-tuning (SFT) and Reinforcement Learning with Human Feedback (RLHF) |
|
Context Length: | |
Training Time: | |
Hardware Used: | H100-80GB GPUs, custom-built GPU clusters |
|
Model Architecture: | Auto-regressive transformer architecture with Grouped-Query Attention (GQA) |
|
|
Safety Evaluation |
Methodologies: | Red teaming, Adversarial evaluation |
|
Findings: | Addressed risks in areas such as CBRNE, child safety, cyber attack enablement |
|
Risk Categories: | Misinformation, Bias, Security threats |
|
Ethical Considerations: | Emphasized responsible use and transparency in deployment. |
|
|
Responsible Ai Considerations |
Fairness: | Implemented safety fine-tuning to mitigate biases. |
|
Transparency: | Open release to community for evaluation and improvement. |
|
Accountability: | Developers responsible for deployment safety. |
|
Mitigation Strategies: | Use of human feedback and LLM-based classifiers for data quality control. |
|
|
Input Output |
Input Format: | |
Accepted Modalities: | |
Output Format: | |
|
Release Notes |
Version: | |
Date: | |
Notes: | Increased contextual length, multilingual expansion, improved safety and performance. |
|
|
|