Model Type | |
Use Cases |
Areas: | commercial applications, research |
|
Applications: | general-purpose AI systems, applications requiring memory/compute constrained environments, latency-bound scenarios, strong reasoning tasks (code, math, logic) |
|
Primary Use Cases: | language and multimodal model research, building blocks for generative AI powered features |
|
Limitations: | not evaluated for all downstream purposes, accuracy, safety, and fairness mitigation required before specific use |
|
Considerations: | Developers should adhere to laws and evaluate accuracy, safety, and fairness before using in high-risk scenarios. |
|
|
Additional Notes | Phi-3.5-MoE is designed for use in constrained environments and supports multilingual capabilities. |
|
Supported Languages | Arabic (supported), Chinese (supported), Czech (supported), Danish (supported), Dutch (supported), English (supported), Finnish (supported), French (supported), German (supported), Hebrew (supported), Hungarian (supported), Italian (supported), Japanese (supported), Korean (supported), Norwegian (supported), Polish (supported), Portuguese (supported), Russian (supported), Spanish (supported), Swedish (supported), Thai (supported), Turkish (supported), Ukrainian (supported) |
|
Training Details |
Data Sources: | publicly available documents, high-quality educational data, synthetic "textbook-like" data, high quality chat format supervised data |
|
Data Volume: | |
Methodology: | supervised fine-tuning, proximal policy optimization, and direct preference optimization |
|
Context Length: | |
Training Time: | |
Hardware Used: | |
Model Architecture: | Mixture-of-Expert decoder-only Transformer model |
|
|
Safety Evaluation |
Methodologies: | red teaming, adversarial conversation simulations, multilingual safety evaluation benchmark datasets |
|
Findings: | Positive impact from safety post-training across multiple languages and risk categories, higher refusal rates for undesirable outputs, robustness to jailbreak techniques., Models may refuse to generate undesirable outputs in English, even when the request is in another language. |
|
Risk Categories: | misinformation, offensive content, multilingual performance and safety gaps |
|
Ethical Considerations: | Ensuring models do not perpetuate harmful stereotypes or generate inappropriate content. |
|
|
Responsible Ai Considerations |
Fairness: | Model may under- or over-represent groups of people or reinforce negative stereotypes due to training data bias. |
|
Transparency: | Developers should inform end-users they are interacting with an AI system. |
|
Accountability: | Developers are responsible for testing for performance or safety gaps and implementing language-specific safeguards. |
|
Mitigation Strategies: | Safety post-training, model fine-tuning, and adherence to legal regulations are recommended. |
|
|
Input Output |
Input Format: | |
Accepted Modalities: | |
Output Format: | |
|