Hummingbird by PeymanHosseini

 ยป  All LLMs  ยป  PeymanHosseini  ยป  Hummingbird   URL Share it on

  Arxiv:2403.01643   Autotrain compatible Dataset:alespalla/chatbot inst...   Dataset:camel-ai/math Dataset:hendrycks/competition ... Dataset:huggingfaceh4/instruct...   Dataset:lighteval/math Dataset:mbzuai/lamini-instruct... Dataset:microsoft/orca-math-wo... Dataset:qwedsacf/grade-school-...   Dataset:wikimedia/wikipedia   En   Endpoints compatible   Hummingbird   Instruct   Region:us   Safetensors

Hummingbird Benchmarks

nn.n% — How the model compares to the reference models: Anthropic Sonnet 3.5 ("so35"), GPT-4o ("gpt4o") or GPT-4 ("gpt4").

Hummingbird Parameters and Internals

Model Type
causal-lm
Additional NotesThis version of Hummingbird is only meant to demonstrate Efficient Attention for use in causal language modelling.
Training Details
Data Sources:
wikimedia/wikipedia, qwedsacf/grade-school-math-instructions, HuggingFaceH4/instruction-dataset, alespalla/chatbot_instruction_prompts, MBZUAI/LaMini-instruction, hendrycks/competition_math, lighteval/MATH, camel-ai/math, microsoft/orca-math-word-problems-200k
Data Volume:15 Billion tokens
Methodology:Efficient Attention
Model Architecture:# Transformer Blocks: 10, Model Dimension: 3072, # Heads: 1
Safety Evaluation
Ethical Considerations:Not safeguarded, not recommended as a chatbot
LLM NameHummingbird
Repository ๐Ÿค—https://huggingface.co/PeymanHosseini/Hummingbird 
Model Size1.1b
Required VRAM2.3 GB
Updated2024-11-13
MaintainerPeymanHosseini
Model Typehummingbird
Instruction-BasedYes
Model Files  2.3 GB
Supported Languagesen
Model ArchitectureHummingbirdForCausalLM
Licensemit
Context Length131072
Model Max Length131072
Transformers Version4.38.2
Tokenizer ClassLlamaTokenizer
Padding Token<s>
Vocabulary Size32000
Torch Data Typebfloat16
Hummingbird (PeymanHosseini/Hummingbird)

Rank the Hummingbird Capabilities

๐Ÿ†˜ Have you tried this model? Rate its performance. This feedback would greatly assist ML community in identifying the most suitable model for their needs. Your contribution really does make a difference! ๐ŸŒŸ

Instruction Following and Task Automation  
Factuality and Completeness of Knowledge  
Censorship and Alignment  
Data Analysis and Insight Generation  
Text Generation  
Text Summarization and Feature Extraction  
Code Generation  
Multi-Language Support and Translation  

What open-source LLMs or SLMs are you in search of? 37901 in total.

Our Social Media →  
Original data from HuggingFace, OpenCompass and various public git repos.
Release v20241110