Jamba V0.1 By ai21labs: Benchmarks, Features and Detailed Analysis. Insights on Jamba V0.1.

Arxiv:2403.19887 Autotrain compatible Custom code Endpoints compatible Jamba Mamba Moe Region:us Safetensors Sharded Tensorflow

Model Card on HF 🤗: https://huggingface.co/ai21labs/Jamba-v0.1

Jamba V0.1 Benchmarks

MMLU Pro: 16.57

GPQA: 2.46

MUSR: 3.71

BBH: 10.72

IFEval: 20.26 vs 88 (so35)^-77%

MATH Lvl 5: 1.13

LLME Score: 0.36967

^nn.n% — How the model compares to the reference models: Anthropic Sonnet 3.5 ("so35"), GPT-4o ("gpt4o") or GPT-4 ("gpt4").

What is the LLM Explorer Rank (Score)

Jamba V0.1 Parameters and Internals

Model Type

Hybrid SSM-Transformer, Mixture-of-Experts (MoE), Generative Text Model

Use Cases

Limitations:

Lacks alignment for instruct/chat interactions

Considerations:

Jamba should have safety moderation mechanisms and guardrails added for responsible and safe use.

Additional Notes

The model is available for fine-tuning using techniques like PEFT.

Training Details

Context Length:

256000

Model Architecture:

Hybrid SSM-Transformer

Input Output

Input Format:

Compatible with 'transformers' API.

Accepted Modalities:

text

Output Format:

Generative text output

Performance Tips:

Use 'transformers' 4.40.0 or higher for optimal performance; install 'mamba-ssm' and 'causal-conv1d' for optimized Mamba implementation

LLM Name	Jamba V0.1
Repository 🤗	https://huggingface.co/ai21labs/Jamba-v0.1
Model Size	51.6b
Required VRAM	103.4 GB
Updated	2024-12-21
Maintainer	ai21labs
Model Type	jamba
Model Files	5.0 GB: 1-of-21 4.9 GB: 2-of-21 5.0 GB: 3-of-21 5.0 GB: 4-of-21 5.0 GB: 5-of-21 4.9 GB: 6-of-21 4.9 GB: 7-of-21 5.0 GB: 8-of-21 4.9 GB: 9-of-21 4.9 GB: 10-of-21 4.9 GB: 11-of-21 4.9 GB: 12-of-21 4.9 GB: 13-of-21 5.0 GB: 14-of-21 4.9 GB: 15-of-21 4.9 GB: 16-of-21 4.9 GB: 17-of-21 4.9 GB: 18-of-21 5.0 GB: 19-of-21 4.9 GB: 20-of-21 4.7 GB: 21-of-21
Model Architecture	JambaForCausalLM
License	apache-2.0
Context Length	262144
Model Max Length	262144
Transformers Version	4.40.1
Tokenizer Class	LlamaTokenizer
Padding Token	<\|pad\|>
Vocabulary Size	65536
Torch Data Type	bfloat16

Quantized Models of the Jamba V0.1

Model	Likes	Downloads	VRAM
Jamba Bagel 4bit	2	15	29 GB

Best Alternatives to Jamba V0.1

Best Alternatives	Context / RAM	Downloads	Likes
Jamba V0.1 Claude Chat	256K / 103.4 GB	15	3
Jamba V0.1 Hf	256K / 103.4 GB	18	3
Jamba V0.1 Chat Multilingual	0K / 103.4 GB	35	22
Code Jamba V0.1	0K / 103.4 GB	29	7
Bagel Jamba V05	0K / 103.6 GB	22	7
Airoboros Jamba 3 3	0K / 103.6 GB	17	6

Rank the Jamba V0.1 Capabilities

🆘 Have you tried this model? Rate its performance. This feedback would greatly assist ML community in identifying the most suitable model for their needs. Your contribution really does make a difference! 🌟

Instruction Following and Task Automation
Factuality and Completeness of Knowledge
Censorship and Alignment
Data Analysis and Insight Generation
Text Generation
Text Summarization and Feature Extraction
Code Generation
Multi-Language Support and Translation

What open-source LLMs or SLMs are you in search of? 40013 in total.

Email us: info@extractum.io. Our Privacy Policy | Terms and Conditions | Suggest an improvement.

Our Social Media →

Original data from HuggingFace, OpenCompass and various public git repos.

Release v20241217

Support LLM Explorer