Camelidae 8x13B by hywu

 ยป  All LLMs  ยป  hywu  ยป  Camelidae 8x13B   URL Share it on

  Arxiv:1902.00751   Arxiv:2212.05055   Arxiv:2305.14314   Arxiv:2401.02731   Autotrain compatible   Camelidae   Custom code Dataset:ise-uiuc/magicoder-evo... Dataset:ise-uiuc/magicoder-oss...   Dataset:meta-math/metamathqa   Dataset:open-orca/slimorca   En   Endpoints compatible   Instruct   Moe   Pytorch   Region:us   Sharded
Model Card on HF ๐Ÿค—: https://huggingface.co/hywu/Camelidae-8x13B 

Camelidae 8x13B Benchmarks

Camelidae 8x13B (hywu/Camelidae-8x13B)

Camelidae 8x13B Parameters and Internals

Model Type 
text-generation
Additional Notes 
Parameter-Efficient Sparsity Crafting utilizes QLoRA and Adapter to perform Efficient Sparse Upcycling.
Supported Languages 
en (English)
Training Details 
Data Sources:
Open-Orca/SlimOrca, ise-uiuc/Magicoder-OSS-Instruct-75K, ise-uiuc/Magicoder-Evol-Instruct-110K, meta-math/MetaMathQA
Methodology:
Parameter-Efficient Sparsity Crafting
Model Architecture:
Mixture-of-Experts
Release Notes 
Version:
Qwen2idae-16x14B-v1.0
Date:
3/12/2024
Notes:
Qwen2idae-16x14B-v1.0 released on Hugging Face with strong performance in Math and Code with 15B activated params.
Version:
Camelidae
Date:
12/22/2023
Notes:
Training repo for crafting dense model with LLaMA architecture to MoE model.
LLM NameCamelidae 8x13B
Repository ๐Ÿค—https://huggingface.co/hywu/Camelidae-8x13B 
Required VRAM29.4โ€ฏGB
Updated2025-01-05
Maintainerhywu
Model Typecamelidae
Instruction-BasedYes
Model Files  9.9 GB: 1-of-3   9.9 GB: 2-of-3   9.6 GB: 3-of-3
Supported Languagesen
Model Architecturemodeling_camelidae.LlamaForCausalLM
Licenseapache-2.0
Context Length4096
Model Max Length4096
Transformers Version4.34.0.dev0
Tokenizer ClassLlamaTokenizer
Beginning of Sentence Token<s>
End of Sentence Token</s>
Unk Token<unk>
Vocabulary Size32000
Torch Data Typebfloat16

Best Alternatives to Camelidae 8x13B

Best Alternatives
Context / RAM
Downloads
Likes
Camelidae 8x7B4K / 15.6โ€‰GB108414
Camelidae 8x34B4K / 76โ€‰GB109728
Note: green Score (e.g. "73.2") means that the model is better than hywu/Camelidae-8x13B.

Rank the Camelidae 8x13B Capabilities

๐Ÿ†˜ Have you tried this model? Rate its performance. This feedback would greatly assist ML community in identifying the most suitable model for their needs. Your contribution really does make a difference! ๐ŸŒŸ

Instruction Following and Task Automation  
Factuality and Completeness of Knowledge  
Censorship and Alignment  
Data Analysis and Insight Generation  
Text Generation  
Text Summarization and Feature Extraction  
Code Generation  
Multi-Language Support and Translation  

What open-source LLMs or SLMs are you in search of? 40918 in total.

Our Social Media โ†’  
Original data from HuggingFace, OpenCompass and various public git repos.
Release v20241227