LLM EXPLORER
 LLM Explorer: A Curated Large Language Model Directory and Analytics
Dark Theme
  LLM  List    LLM Hosting    LLM Leaderboards    Blog    Newsfeed    Advertise

Mpt 30B Instruct Q8 by Abzu

 »  All LLMs  »  Abzu  »  Mpt 30B Instruct Q8   URL Share it on

  16   
  Arxiv:2108.12409   Arxiv:2205.14135   8-bit   Autotrain compatible   Composer   Custom code   Endpoints compatible   Instruct   Llm-foundry   Mosaicml   Mpt   Q8   Quantized   Region:us   Safetensors   Sharded   Tensorflow
Model Card on HF 🤗: https://huggingface.co/Abzu/mpt-30b-instruct-q8 

Mpt 30B Instruct Q8 Benchmarks

ARC: 58.45 vs 96.7 (so35)-39.6%
HellaSwag: 84.31 vs 95.3 (gpt4)-11.5%
MMLU: 49.15 vs 88.3 (so35)-44.3%
TruthfulQA: 38.05 vs 59 (gpt4)-35.5%
WinoGrande: 75.14 vs 87.5 (gpt4)-14.1%
GSM8K: 15.31 vs 96.4 (so35)-84.1%
LLME Score: 0.14518
nn.n% — How the model compares to the reference models: Anthropic Sonnet 3.5 ("so35"), GPT-4o ("gpt4o") or GPT-4 ("gpt4").
Mpt 30B Instruct Q8 (Abzu/mpt-30b-instruct-q8)

Mpt 30B Instruct Q8 Parameters and Internals

Model Type 
text generation, instruction following
Training Details 
Data Sources:
competition_math, cot_gsm8k, dialogsum, dolly_hhrlhf, duorc, qasper, quality, scrolls/summ_screen_fd, spider
Context Length:
8192
Training Time:
8 hours
Hardware Used:
72 A100 40GB GPUs
Model Architecture:
Modified decoder-only transformer with FlashAttention, ALiBi, no biases
LLM NameMpt 30B Instruct Q8
Repository 🤗https://huggingface.co/Abzu/mpt-30b-instruct-q8 
Base Model(s)  Mpt 30B Instruct   mosaicml/mpt-30b-instruct
Model Size30b
Required VRAM30.4 GB
Updated2024-12-22
MaintainerAbzu
Model Typempt
Instruction-BasedYes
Model Files  10.0 GB: 1-of-4   9.9 GB: 2-of-4   9.9 GB: 3-of-4   0.6 GB: 4-of-4
Quantization Typeq8
Model ArchitectureMPTForCausalLM
Licensecc-by-sa-3.0
Transformers Version4.30.2
Tokenizer ClassGPTNeoXTokenizer
Vocabulary Size50432
Torch Data Typebfloat16

Best Alternatives to Mpt 30B Instruct Q8

Best Alternatives
Context / RAM
Downloads
Likes
Mpt 30B Chat Q80K / 30.4 GB191
Mpt 30B Instruct0K / 60.1 GB1263101
...l Mpt 30B Instruct W4 G128 AWQ0K / 16.1 GB82
...t 30B Instruct Peft Compatible0K / 60.1 GB132
Ct2fast Mpt 30B Instruct0K / 30 GB94
Ct2fast Mpt 30B Chat0K / 30 GB102
Note: green Score (e.g. "73.2") means that the model is better than Abzu/mpt-30b-instruct-q8.

Rank the Mpt 30B Instruct Q8 Capabilities

🆘 Have you tried this model? Rate its performance. This feedback would greatly assist ML community in identifying the most suitable model for their needs. Your contribution really does make a difference! 🌟

Instruction Following and Task Automation  
Factuality and Completeness of Knowledge  
Censorship and Alignment  
Data Analysis and Insight Generation  
Text Generation  
Text Summarization and Feature Extraction  
Code Generation  
Multi-Language Support and Translation  

What open-source LLMs or SLMs are you in search of? 40066 in total.

Email us: info@extractum.io. Our Privacy Policy  | Terms and Conditions  | Suggest an improvement.
Our Social Media →  
Original data from HuggingFace, OpenCompass and various public git repos.
Release v20241217
  Support LLM Explorer