Model Type | text generation, chat format |
|
Use Cases |
Areas: | |
Applications: | memory/compute constrained environments, latency bound scenarios, strong reasoning, long context |
|
Primary Use Cases: | acceleration of research on language and multimodal models, building generative AI features |
|
Limitations: | Not specifically designed or evaluated for all downstream purposes. |
|
Considerations: | Developers should adhere to laws, mitigate against bias and inaccuracies. |
|
|
Additional Notes | Model is well-suited for research and generative AI applications with focus on strong reasoning capabilities and long context. |
|
Supported Languages | |
Training Details |
Data Sources: | Phi-3 datasets, synthetic data, filtered publicly available websites |
|
Data Volume: | |
Methodology: | Supervised fine-tuning and Direct Preference Optimization |
|
Context Length: | |
Training Time: | |
Hardware Used: | |
Model Architecture: | Dense decoder-only Transformer |
|
|
Safety Evaluation |
Methodologies: | Post-training supervised fine-tuning and direct preference optimization for safety. |
|
Findings: | Unfairness, unreliability, or offensive content may still be present despite safety post-training. |
|
Risk Categories: | Quality of Service, Representation of Harms & Stereotypes, Inappropriate/Offensive Content, Information Reliability, Limited Scope for Code |
|
Ethical Considerations: | Developers should evaluate safety and fairness before using in high risk scenarios. |
|
|
Responsible Ai Considerations |
Fairness: | Model may over- or under-represent groups or reinforce stereotypes. |
|
Transparency: | Developers should inform end-users that they are interacting with an AI system. |
|
Accountability: | Developers are responsible for ensuring compliant use in specific scenarios. |
|
Mitigation Strategies: | Consider transparency and mitigate risks in high-risk scenarios. |
|
|
Input Output |
Input Format: | Chat format (e.g., <|user|> prompt format). |
|
Accepted Modalities: | |
Output Format: | Generated text in response to input. |
|
Performance Tips: | Provide inputs in chat format for best results. |
|
|
Release Notes |
Version: | |
Date: | |
Notes: | Trained between February and April 2024, on 3.3T tokens. |
|
|
|