Mixtral 8x7B Instruct V0.1 FP8 V2 by comaniac

 ยป  All LLMs  ยป  comaniac  ยป  Mixtral 8x7B Instruct V0.1 FP8 V2   URL Share it on

  Autotrain compatible   Conversational   Endpoints compatible   Fp8   Instruct   Mixtral   Moe   Region:us   Safetensors   Sharded   Tensorflow

Mixtral 8x7B Instruct V0.1 FP8 V2 Benchmarks

nn.n% — How the model compares to the reference models: Anthropic Sonnet 3.5 ("so35"), GPT-4o ("gpt4o") or GPT-4 ("gpt4").
Mixtral 8x7B Instruct V0.1 FP8 V2 (comaniac/Mixtral-8x7B-Instruct-v0.1-FP8-v2)

Mixtral 8x7B Instruct V0.1 FP8 V2 Parameters and Internals

Additional Notes 
Weights and activations are per-tensor quantized to float8_e4m3.
Training Details 
Data Volume:
Ultrachat-200k
Methodology:
Quantization with AutoFP8
Context Length:
8192
LLM NameMixtral 8x7B Instruct V0.1 FP8 V2
Repository ๐Ÿค—https://huggingface.co/comaniac/Mixtral-8x7B-Instruct-v0.1-FP8-v2 
Model Size46.7b
Required VRAM47.1 GB
Updated2025-02-09
Maintainercomaniac
Model Typemixtral
Instruction-BasedYes
Model Files  5.0 GB: 1-of-10   5.0 GB: 2-of-10   5.0 GB: 3-of-10   5.0 GB: 4-of-10   5.0 GB: 5-of-10   5.0 GB: 6-of-10   5.0 GB: 7-of-10   5.0 GB: 8-of-10   5.0 GB: 9-of-10   2.1 GB: 10-of-10
Model ArchitectureMixtralForCausalLM
Context Length32768
Model Max Length32768
Transformers Version4.40.2
Tokenizer ClassLlamaTokenizer
Vocabulary Size32000
Torch Data Typebfloat16

Best Alternatives to Mixtral 8x7B Instruct V0.1 FP8 V2

Best Alternatives
Context / RAM
Downloads
Likes
Mixtral 8x7B Instruct V0.132K / 93.6 GB6433814290
Dolphin 2.5 Mixtral 8x7b32K / 93.6 GB144671226
Merge Mixtral Prometheus 8x7B32K / 91.9 GB102
...rkrautLM Mixtral 8x7B Instruct32K / 93.6 GB166322
Notux 8x7b V132K / 93.6 GB58165
BagelMIsteryTour V2 8x7B32K / 93.5 GB5316
Karakuri Lm 8x7b Instruct V0.132K / 93.6 GB11319
Autotrain Xva0j Mixtral8x7b32K / 93.6 GB490
Mixtral 8x7B Instruct V0.1 FP832K / 47.1 GB6293
XLAM V0.1 R32K / 93.6 GB15952
Note: green Score (e.g. "73.2") means that the model is better than comaniac/Mixtral-8x7B-Instruct-v0.1-FP8-v2.

Rank the Mixtral 8x7B Instruct V0.1 FP8 V2 Capabilities

๐Ÿ†˜ Have you tried this model? Rate its performance. This feedback would greatly assist ML community in identifying the most suitable model for their needs. Your contribution really does make a difference! ๐ŸŒŸ

Instruction Following and Task Automation  
Factuality and Completeness of Knowledge  
Censorship and Alignment  
Data Analysis and Insight Generation  
Text Generation  
Text Summarization and Feature Extraction  
Code Generation  
Multi-Language Support and Translation  

What open-source LLMs or SLMs are you in search of? 42884 in total.

Our Social Media →  
Original data from HuggingFace, OpenCompass and various public git repos.
Release v20241227