Model Type | Causal decoder-only transformer language model |
|
Use Cases |
Applications: | Writing, Explanation, Discussion tasks |
|
Primary Use Cases: | Bilingual chat in English and German |
|
Limitations: | Struggles with math and advanced reasoning |
|
Considerations: | Developers should perform safety testing prior to deployment. |
|
|
Additional Notes | Developed in collaboration with LAION and HessianAI. |
|
Supported Languages | en (Bilingual support in English and German), de (Bilingual support in English and German) |
|
Training Details |
Data Sources: | LeoLM/OpenSchnabeltier, OpenAssistant/OASST-DE, FreedomIntelligence/alpaca-gpt4-deutsch, FreedomIntelligence/evol-instruct-deutsch, LeoLM/German_Poems, LeoLM/German_Songs, garage-bAInd/Open-Platypus, WizardLM/WizardLM_evol_instruct_70k, bjoernp/oasst25-08-23-filtered |
|
Data Volume: | Total Accepted: 235632 samples
Total Accepted tokens: 115,862,397 |
|
Methodology: | Continued pretraining on a large corpus of German-language text |
|
Context Length: | |
Hardware Used: | HessianAI's new supercomputer 42 |
|
|
Safety Evaluation |
Ethical Considerations: | Potential outputs cannot be predicted in advance, may include inaccurate, biased, or objectionable responses. |
|
|
Input Output |
Input Format: | |
Accepted Modalities: | |
Output Format: | |
Performance Tips: | Can utilize flash-attn2 for faster inference. |
|
|