LLM EXPLORER
 LLM Explorer: A Curated Large Language Model Directory and Analytics
Dark Theme
  LLM  List    LLM Hosting    LLM Leaderboards    Blog    Newsfeed    Advertise

Falcon Rw 7B by tiiuae

 »  All LLMs  »  tiiuae  »  Falcon Rw 7B   URL Share it on

  2212   
  Arxiv:2005.14165   Arxiv:2108.12409   Arxiv:2205.14135   Arxiv:2306.01116   Autotrain compatible   Custom code Dataset:tiiuae/falcon-refinedw...   En   Falcon   Pytorch   Region:us   Safetensors   Sharded   Tensorflow
Model Card on HF 🤗: https://huggingface.co/tiiuae/falcon-rw-7b 

Falcon Rw 7B Benchmarks

LLME Score: 0.11753
nn.n% — How the model compares to the reference models: Anthropic Sonnet 3.5 ("so35"), GPT-4o ("gpt4o") or GPT-4 ("gpt4").
Falcon Rw 7B (tiiuae/falcon-rw-7b)

Falcon Rw 7B Parameters and Internals

Model Type 
causal decoder-only
Use Cases 
Primary Use Cases:
Research on large language models, specifically on training models on filtered web data
Limitations:
Production use without adequate risk assessment may be irresponsible or harmful, Not suitable for non-English data due to training on English data only
Considerations:
Finetuning and guardrail setups for production contexts.
Additional Notes 
Falcon is released under the Apache 2.0 license and is intended as a research artifact.
Supported Languages 
English (native)
Training Details 
Data Sources:
RefinedWeb
Data Volume:
350B tokens
Methodology:
Training adapted from GPT-3 paper using ALiBi and FlashAttention
Context Length:
2048
Training Time:
approximately five days
Hardware Used:
256 A100 40GB GPUs
Model Architecture:
36 layers, d_model=4096, head_dim=64
Responsible Ai Considerations 
Fairness:
Training on large-scale web data might introduce common online stereotypes and biases.
Mitigation Strategies:
Finetuning the model for specific tasks and setting appropriate guardrails for production use.
LLM NameFalcon Rw 7B
Repository 🤗https://huggingface.co/tiiuae/falcon-rw-7b 
Model Size7b
Required VRAM15.5 GB
Updated2025-02-22
Maintainertiiuae
Model Typefalcon
Model Files  9.9 GB: 1-of-2   5.6 GB: 2-of-2   9.9 GB: 1-of-2   5.6 GB: 2-of-2
Supported Languagesen
Model ArchitectureFalconForCausalLM
Licenseapache-2.0
Model Max Length2048
Transformers Version4.27.4
Is Biased1
Tokenizer ClassPreTrainedTokenizerFast
Vocabulary Size65024
Torch Data Typebfloat16

Best Alternatives to Falcon Rw 7B

Best Alternatives
Context / RAM
Downloads
Likes
F1H10M 00002K / 13.9 GB25290
F1H10M 00002K / 13.9 GB20120
Really Tiny Falcon Testing2K / 0 GB3368790
Saqr 7B Merged2K / 13.9 GB1431
Tiny Testing Falcon Alibi2K / 0 GB291661
... Openassistant Toxicity Reduce2K / 30.4 GB50
Falcon Chatbot2K / 5.5 GB531
DociproLLM 7B2K / 27.8 GB80
New Falcon2K / 4.3 GB1621
Claire 7B 0.1 Instruct2K / 7.2 GB1051
Note: green Score (e.g. "73.2") means that the model is better than tiiuae/falcon-rw-7b.

Rank the Falcon Rw 7B Capabilities

🆘 Have you tried this model? Rate its performance. This feedback would greatly assist ML community in identifying the most suitable model for their needs. Your contribution really does make a difference! 🌟

Instruction Following and Task Automation  
Factuality and Completeness of Knowledge  
Censorship and Alignment  
Data Analysis and Insight Generation  
Text Generation  
Text Summarization and Feature Extraction  
Code Generation  
Multi-Language Support and Translation  

What open-source LLMs or SLMs are you in search of? 43470 in total.

Email us: info@extractum.io. Our Privacy Policy  | Terms and Conditions  | Suggest an improvement.
Our Social Media →  
Original data from HuggingFace, OpenCompass and various public git repos.
Release v20241227
  Support LLM Explorer