CroissantLLMBase by croissantllm

 ยป  All LLMs  ยป  croissantllm  ยป  CroissantLLMBase   URL Share it on

  Arxiv:2402.00786   Art   Autotrain compatible   Code   Dataset:bigcode/starcoderdata Dataset:cerebras/slimpajama-62... Dataset:croissantllm/croissant...   Dataset:pg19   Dataset:uonlp/culturax   En   Endpoints compatible   Fr   Legal   Llama   Pytorch   Region:us   Sharded

CroissantLLMBase Benchmarks

CroissantLLMBase (croissantllm/CroissantLLMBase)

CroissantLLMBase Parameters and Internals

Model Type 
text generation
Additional Notes 
The final model is suited for few-shot prompting strategies and base functions, not finetuned for chat capabilities.
Supported Languages 
en (proficient), fr (proficient)
Training Details 
Data Sources:
cerebras/SlimPajama-627B, uonlp/CulturaX, pg19, bigcode/starcoderdata, croissantllm/croissant_dataset
Data Volume:
3T tokens in English and French
Methodology:
Bilingual training with a custom tokenizer and bilingual finetuning datasets
LLM NameCroissantLLMBase
Repository ๐Ÿค—https://huggingface.co/croissantllm/CroissantLLMBase 
Model Size627b
Required VRAM5.4 GB
Updated2025-01-15
Maintainercroissantllm
Model Typellama
Model Files  5.0 GB: 1-of-2   0.4 GB: 2-of-2
Supported Languagesfr en
Model ArchitectureLlamaForCausalLM
Licensemit
Context Length2048
Model Max Length2048
Transformers Version4.35.0.dev0
Tokenizer ClassLlamaTokenizer
Vocabulary Size32000
Torch Data Typefloat32

Best Alternatives to CroissantLLMBase

Best Alternatives
Context / RAM
Downloads
Likes
TinyLlama V1.12K / 4.4 GB5278680
MicroLlama2K / 1.2 GB154642
TinyLlama V1.1 Math Code2K / 4.4 GB150910
TinyLlama 1.1B 1T OpenOrca2K / 2.2 GB2577
...Llama 1.1B 1.5T OpenOrca Alpha2K / 2.2 GB734
TinyLlama V1.1 Chinese2K / 4.4 GB1637
...inyLlama 1.1B 1T OpenOrca GPTQ2K / 0.8 GB172
TinyLlama 1.1B 1T OpenOrca AWQ2K / 0.8 GB162
Note: green Score (e.g. "73.2") means that the model is better than croissantllm/CroissantLLMBase.

Rank the CroissantLLMBase Capabilities

๐Ÿ†˜ Have you tried this model? Rate its performance. This feedback would greatly assist ML community in identifying the most suitable model for their needs. Your contribution really does make a difference! ๐ŸŒŸ

Instruction Following and Task Automation  
Factuality and Completeness of Knowledge  
Censorship and Alignment  
Data Analysis and Insight Generation  
Text Generation  
Text Summarization and Feature Extraction  
Code Generation  
Multi-Language Support and Translation  

What open-source LLMs or SLMs are you in search of? 41362 in total.

Our Social Media →  
Original data from HuggingFace, OpenCompass and various public git repos.
Release v20241227