Gemma 2 9B Instruct 4Bit GPTQ by Granther

 ยป  All LLMs  ยป  Granther  ยป  Gemma 2 9B Instruct 4Bit GPTQ   URL Share it on

  4-bit   4bit   Autotrain compatible Base model:google/gemma-2-9b-i...   Conversational   Gemma2   Gptq   Instruct   License:gemma   Quantized   Region:us   Safetensors   Sharded   Tensorflow

Gemma 2 9B Instruct 4Bit GPTQ Parameters and Internals

LLM NameGemma 2 9B Instruct 4Bit GPTQ
RepositoryOpen on ๐Ÿค— 
Model NameGemma-2-9B-Instruct-4Bit-GPTQ
Base Model(s)  Gemma 2 9B It   google/gemma-2-9b-it
Model Size9b
Required VRAM6.2 GB
Model Typegemma2
Model Files  5.0 GB: 1-of-2   1.2 GB: 2-of-2
GPTQ QuantizationYes
Quantization Typegptq|4bit
Model ArchitectureGemma2ForCausalLM
Context Length8192
Model Max Length8192
Transformers Version4.43.0.dev0
Tokenizer ClassGemmaTokenizer
Padding Token<pad>
Vocabulary Size256000
Torch Data Typefloat16
Gemma 2 9B Instruct 4Bit GPTQ (Granther/Gemma-2-9B-Instruct-4Bit-GPTQ)

Rank the Gemma 2 9B Instruct 4Bit GPTQ Capabilities

๐Ÿ†˜ Have you tried this model? Rate its performance. This feedback would greatly assist ML community in identifying the most suitable model for their needs. Your contribution really does make a difference! ๐ŸŒŸ

Instruction Following and Task Automation  
Factuality and Completeness of Knowledge  
Censorship and Alignment  
Data Analysis and Insight Generation  
Text Generation  
Text Summarization and Feature Extraction  
Code Generation  
Multi-Language Support and Translation  

What open-source LLMs or SLMs are you in search of? 34211 in total.

Our Social Media →  
Original data from HuggingFace, OpenCompass and various public git repos.
Release v2024071601