Transformer Lm Japanese 0.1B by fukugawa

 ยป  All LLMs  ยป  fukugawa  ยป  Transformer Lm Japanese 0.1B   URL Share it on

  Autotrain compatible   Custom code   Dataset:wiki40b   Endpoints compatible   Flax   Ja   Japanese   Jax   Lm   Lm1b   Region:us   Transformerlm

Transformer Lm Japanese 0.1B Benchmarks

nn.n% — How the model compares to the reference models: Anthropic Sonnet 3.5 ("so35"), GPT-4o ("gpt4o") or GPT-4 ("gpt4").
Transformer Lm Japanese 0.1B (fukugawa/transformer-lm-japanese-0.1b)

Transformer Lm Japanese 0.1B Parameters and Internals

Model Type 
text-generation, lm
Supported Languages 
ja (native)
Training Details 
Data Sources:
wiki40b/ja
Data Volume:
2.19GB
Methodology:
Modified Flax's 'lm1b' example to train on Japanese dataset.
Training Time:
1.5 days
Hardware Used:
CPU for testing, GPU for model training (not specifically mentioned)
Model Architecture:
Transformer
Input Output 
Input Format:
tokenized Japanese text
Accepted Modalities:
text
Output Format:
Generated Japanese text
Performance Tips:
Set 'trust_remote_code=True' to use the custom model code.
Release Notes 
Version:
2024/05/20
Notes:
Added JGLUE 4-task benchmark scores.
Version:
2024/05/13
Notes:
FlaxAutoModelForCausalLM is now supported with custom model code added.
LLM NameTransformer Lm Japanese 0.1B
Repository ๐Ÿค—https://huggingface.co/fukugawa/transformer-lm-japanese-0.1b 
Model Size0.1b
Updated2025-02-22
Maintainerfukugawa
Model Typetransformerlm
Supported Languagesja
Model ArchitectureTransformerLMForCausalLM
Licenseapache-2.0
Transformers Version4.39.0
Tokenizer ClassTransformerLMTokenizer
Padding Token<pad>
Vocabulary Size30000

Rank the Transformer Lm Japanese 0.1B Capabilities

๐Ÿ†˜ Have you tried this model? Rate its performance. This feedback would greatly assist ML community in identifying the most suitable model for their needs. Your contribution really does make a difference! ๐ŸŒŸ

Instruction Following and Task Automation  
Factuality and Completeness of Knowledge  
Censorship and Alignment  
Data Analysis and Insight Generation  
Text Generation  
Text Summarization and Feature Extraction  
Code Generation  
Multi-Language Support and Translation  

What open-source LLMs or SLMs are you in search of? 43470 in total.

Our Social Media →  
Original data from HuggingFace, OpenCompass and various public git repos.
Release v20241227