Model Type | Mixture of Experts (MoE), Bilingual |
|
Use Cases |
Areas: | Research, Commercial Applications |
|
Limitations: | |
Considerations: | Engagement with the community for guardrails |
|
|
Additional Notes | The AquilaMoE Instruct model is a demonstration of easy fine-tuning for compelling performance |
|
Supported Languages | |
Training Details |
Data Sources: | RedPajama-Data-V2, falcon-refinedweb, C4, Pile, WuDaoCorporaText, ChineseWebText |
|
Data Volume: | |
Methodology: | EfficientScale with Scale-Up and Scale-Out strategies |
|
Context Length: | |
|