LLM EXPLORER
 LLM Explorer: A Curated Large Language Model Directory and Analytics
Dark Theme
  LLM  List    LLM Hosting    LLM Leaderboards    Blog    Newsfeed    Advertise

Falcon Mamba 7B by tiiuae

 »  All LLMs  »  tiiuae  »  Falcon Mamba 7B   URL Share it on

  12307   
  Arxiv:2312.00752   Arxiv:2410.05355   Autotrain compatible Dataset:huggingfacefw/fineweb-... Dataset:tiiuae/falcon-refinedw...   En   Endpoints compatible   Falcon mamba   Model-index   Region:us   Safetensors   Sharded   Tensorflow
Model Card on HF 🤗: https://huggingface.co/tiiuae/falcon-mamba-7b 

Falcon Mamba 7B Benchmarks

MMLU Pro: 14.47
GPQA: 8.05
MUSR: 10.86
BBH: 19.88
IFEval: 33.36 vs 88 (so35)-62.1%
MATH Lvl 5: 4.08
LLME Score: 0.37155
nn.n% — How the model compares to the reference models: Anthropic Sonnet 3.5 ("so35"), GPT-4o ("gpt4o") or GPT-4 ("gpt4").
Falcon Mamba 7B (tiiuae/falcon-mamba-7b)

Falcon Mamba 7B Parameters and Internals

Model Type 
Causal decoder-only
Use Cases 
Areas:
intended for text generation
Supported Languages 
English (Main language)
Training Details 
Data Sources:
tiiuae/falcon-refinedweb, HuggingFaceFW/fineweb-edu
Data Volume:
~ 5,500 GT
Methodology:
multi-stage training to increase context-length, Curriculum Learning
Context Length:
8192
Training Time:
roughly two months
Hardware Used:
256 H100 80GB GPUs
Model Architecture:
Mamba architecture
Input Output 
Performance Tips:
Install optimized Mamba kernels with 'causal-conv1d>=1.4.0' and 'mamba-ssm'.
LLM NameFalcon Mamba 7B
Repository 🤗https://huggingface.co/tiiuae/falcon-mamba-7b 
Model Size7b
Required VRAM14.6 GB
Updated2024-12-22
Maintainertiiuae
Model Typefalcon_mamba
Model Files  5.0 GB: 1-of-3   5.0 GB: 2-of-3   4.6 GB: 3-of-3
Supported Languagesen
Model ArchitectureFalconMambaForCausalLM
Licenseother
Transformers Version4.43.0.dev0
Tokenizer ClassPreTrainedTokenizerFast
Vocabulary Size65024
Torch Data Typebfloat16

Quantized Models of the Falcon Mamba 7B

Model
Likes
Downloads
VRAM
Falcon Mamba 7B 4bit11594 GB

Best Alternatives to Falcon Mamba 7B

Best Alternatives
Context / RAM
Downloads
Likes
Falcon3 Mamba 7B Instruct0K / 14.6 GB60516
Falcon3 Mamba 7B Base0K / 14.6 GB10515
Falcon Mamba 7B Instruct0K / 14.6 GB1276567
Falcon Mamba 7B 4bit0K / 4.9 GB5911
Note: green Score (e.g. "73.2") means that the model is better than tiiuae/falcon-mamba-7b.

Rank the Falcon Mamba 7B Capabilities

🆘 Have you tried this model? Rate its performance. This feedback would greatly assist ML community in identifying the most suitable model for their needs. Your contribution really does make a difference! 🌟

Instruction Following and Task Automation  
Factuality and Completeness of Knowledge  
Censorship and Alignment  
Data Analysis and Insight Generation  
Text Generation  
Text Summarization and Feature Extraction  
Code Generation  
Multi-Language Support and Translation  

What open-source LLMs or SLMs are you in search of? 40066 in total.

Email us: info@extractum.io. Our Privacy Policy  | Terms and Conditions  | Suggest an improvement.
Our Social Media →  
Original data from HuggingFace, OpenCompass and various public git repos.
Release v20241217
  Support LLM Explorer