Model Type | Transformer-based Language Model, causal-lm |
|
Use Cases |
Areas: | Research, Scientific uses |
|
Primary Use Cases: | Extract features useful for downstream tasks |
|
Limitations: | Not intended for deployment as-is, Not fine-tuned for downstream tasks such as writing genre prose or commercial chatbots, English-language only |
|
Considerations: | Use in accordance with the Apache 2.0 license. Conduct your own risk and bias assessment when fine-tuning for specific tasks. |
|
|
Additional Notes | The model was trained on 'the Pile', which was not deduplicated before use. Contains texts with biases regarding gender, religion, and race. Curating outputs before presenting to a human reader is recommended. |
|
Supported Languages | |
Training Details |
Data Sources: | |
Methodology: | Autoregressive training using GPT-NeoX library |
|
Context Length: | |
Model Architecture: | Architecture resembles GPT-3 and is almost identical to that of GPT-J-6B. |
|
|