LLM Name | Tiiuae Falcon 40B Instruct W4 G128 AWQ |
Repository ๐ค | https://huggingface.co/abhinavkulkarni/tiiuae-falcon-40b-instruct-w4-g128-awq |
Model Size | 40b |
Required VRAM | 22.3 GB |
Updated | 2024-09-19 |
Maintainer | abhinavkulkarni |
Model Type | RefinedWeb |
Instruction-Based | Yes |
Model Files | |
AWQ Quantization | Yes |
Quantization Type | awq |
Model Architecture | RWForCausalLM |
License | apache-2.0 |
Model Max Length | 2048 |
Transformers Version | 4.33.1 |
Is Biased | 0 |
Tokenizer Class | PreTrainedTokenizerFast |
Vocabulary Size | 65024 |
Torch Data Type | float16 |
Best Alternatives |
Context / RAM |
Downloads |
Likes |
---|---|---|---|
Falcon 40B Instruct 8bit | 0K / 41.8 GB | 11 | 6 |
Falcon 40B Instruct GPTQ | 0K / 22.5 GB | 29 | 198 |
H2ogpt Oig Oasst1 Falcon 40B | 0K / 82.5 GB | 71 | 6 |
Falcon 40B Instruct GPTQ | 0K / 22.5 GB | 6 | 1 |
...truct GPTQ Inference Endpoints | 0K / 22.5 GB | 9 | 2 |
๐ Have you tried this model? Rate its performance. This feedback would greatly assist ML community in identifying the most suitable model for their needs. Your contribution really does make a difference! ๐