LLM EXPLORER
 LLM Explorer: A Curated Large Language Model Directory and Analytics
Dark Theme
  LLM  List    LLM Hosting    LLM Leaderboards    Blog    Newsfeed    Advertise

MixtureofMerges MoE 4x7b V4 by jsfs11

 »  All LLMs  »  jsfs11  »  MixtureofMerges MoE 4x7b V4   URL Share it on

  4288   
  Autotrain compatible Base model:flemmingmiguel/mbx-... Base model:kukedlc/neutrixomni... Base model:merge:flemmingmigue... Base model:merge:kukedlc/neutr... Base model:merge:petrogpt/west... Base model:merge:vanillaovo/su... Base model:petrogpt/westseveru... Base model:vanillaovo/supermar...   Endpoints compatible   Flemmingmiguel/mbx-7b-v3   Frankenmoe Kukedlc/neutrixomnibe-7b-model...   Lazymergekit   Merge   Mergekit   Mixtral   Model-index   Moe   Petrogpt/westseverus-7b-dpo   Region:us   Safetensors   Sharded   Tensorflow   Vanillaovo/supermario v4

MixtureofMerges MoE 4x7b V4 Benchmarks

MMLU Pro: 22.58
GPQA: 4.81
MUSR: 13.89
BBH: 32.22
IFEval: 40.3 vs 88 (so35)-54.2%
ARC: 72.53 vs 96.7 (so35)-25%
HellaSwag: 88.85 vs 95.3 (gpt4)-6.8%
MMLU: 64.53 vs 88.3 (so35)-26.9%
TruthfulQA: 75.3 vs 59 (gpt4)27.6%
WinoGrande: 84.85 vs 87.5 (gpt4)-3%
GSM8K: 71.34 vs 96.4 (so35)-26%
MATH Lvl 5: 5.74
LLME Score: 0.30621
nn.n% — How the model compares to the reference models: Anthropic Sonnet 3.5 ("so35"), GPT-4o ("gpt4o") or GPT-4 ("gpt4").

MixtureofMerges MoE 4x7b V4 Parameters and Internals

LLM NameMixtureofMerges MoE 4x7b V4
Repository 🤗https://huggingface.co/jsfs11/MixtureofMerges-MoE-4x7b-v4 
Base Model(s)  MBX 7B V3   NeuTrixOmniBe 7B Model Remix   WestSeverus 7B DPO   Supermario V4   flemmingmiguel/MBX-7B-v3   Kukedlc/NeuTrixOmniBe-7B-model-remix   PetroGPT/WestSeverus-7B-DPO   vanillaOVO/supermario_v4
Model Size24.2b
Required VRAM48.3 GB
Updated2024-09-16
Maintainerjsfs11
Model Typemixtral
Model Files  9.9 GB: 1-of-5   10.0 GB: 2-of-5   10.0 GB: 3-of-5   10.0 GB: 4-of-5   8.4 GB: 5-of-5
Model ArchitectureMixtralForCausalLM
Licenseapache-2.0
Context Length32768
Model Max Length32768
Transformers Version4.37.2
Tokenizer ClassLlamaTokenizer
Padding Token<s>
Vocabulary Size32000
Torch Data Typebfloat16
MixtureofMerges MoE 4x7b V4 (jsfs11/MixtureofMerges-MoE-4x7b-v4)

Best Alternatives to MixtureofMerges MoE 4x7b V4

Best Alternatives
Context / RAM
Downloads
Likes
Dzakwan MoE 4x7b Beta32K / 48.4 GB42180
Beyonder 4x7B V332K / 48.3 GB432557
Mera Mix 4x7B32K / 48.3 GB483018
Calme 4x7B MoE V0.232K / 48.3 GB48842
Calme 4x7B MoE V0.132K / 48.3 GB42732
MixtureofMerges MoE 4x7b V532K / 48.3 GB42941
CognitiveFusion2 4x7B BF1632K / 48.3 GB48813
NeuralStar FusionWriter 4x7b32K / 48.3 GB215
LCARS AI 1x4 003 SuperAI32K / 48.5 GB552
Mixtral 4x7b Slerp32K / 96.8 GB11
Note: green Score (e.g. "73.2") means that the model is better than jsfs11/MixtureofMerges-MoE-4x7b-v4.

Rank the MixtureofMerges MoE 4x7b V4 Capabilities

🆘 Have you tried this model? Rate its performance. This feedback would greatly assist ML community in identifying the most suitable model for their needs. Your contribution really does make a difference! 🌟

Instruction Following and Task Automation  
Factuality and Completeness of Knowledge  
Censorship and Alignment  
Data Analysis and Insight Generation  
Text Generation  
Text Summarization and Feature Extraction  
Code Generation  
Multi-Language Support and Translation  

What open-source LLMs or SLMs are you in search of? 35926 in total.

Email us: info@extractum.io. Our Privacy Policy  | Terms and Conditions  | Suggest an improvement.
Our Social Media →  
Original data from HuggingFace, OpenCompass and various public git repos.
Release v2024072803
  Support LLM Explorer