Model Type | auto-regressive, transformer |
|
Use Cases |
Areas: | |
Applications: | question answering, natural language understanding, reading comprehension |
|
Primary Use Cases: | Developing techniques to improve language models |
|
Limitations: | Should not be used for downstream applications without risk evaluation |
|
Considerations: | The model can generate toxic content, incorrect information, or unhelpful answers due to lack of human feedback training. |
|
|
Additional Notes | Requires special support code and converted via GPTQ method, experimental release. |
|
Supported Languages | en (English), bg (Bulgarian), ca (Catalan), cs (Czech), da (Danish), de (German), es (Spanish), fr (French), hr (Croatian), hu (Hungarian), it (Italian), nl (Dutch), pl (Polish), pt (Portuguese), ro (Romanian), ru (Russian), sl (Slovene), sr (Serbian), sv (Swedish), uk (Ukrainian) |
|
Training Details |
Data Sources: | CCNet, C4, GitHub, Wikipedia, Books, ArXiv, Stack Exchange |
|
Model Architecture: | |
|
Safety Evaluation |
Risk Categories: | misinformation, bias, toxicity |
|
Ethical Considerations: | The model reflects biases from its training data, which is collected from various web sources. |
|
|
Responsible Ai Considerations |
Fairness: | Evaluated on RAI datasets for gender, religion, race, sexual orientation, age, nationality, disability, physical appearance, and socio-economic status biases. |
|
Transparency: | Bias evaluation results are reported. |
|
Mitigation Strategies: | Filtered web data based on its proximity to Wikipedia using Kneser-Ney and fastText linear classifier. |
|
|
Input Output | |