Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -23,7 +23,7 @@ This is a small multilingual language model based on a Transformer architecture
|
|
| 23 |
|
| 24 |
### Architecture
|
| 25 |
|
| 26 |
-
- Transformer-based language model (Decoder-only).
|
| 27 |
- Reduced model dimensions (`n_embd=768`, `n_head=12`, `n_layer=12`) for faster training and smaller model size, making it suitable for resource-constrained environments.
|
| 28 |
- Uses Byte-Pair Encoding (BPE) tokenizer trained on the same Wikipedia data.
|
| 29 |
|
|
|
|
| 23 |
|
| 24 |
### Architecture
|
| 25 |
|
| 26 |
+
- Transformer-based language model (Decoder-only), now using Hugging Face Transformers' GPT2 architecture.
|
| 27 |
- Reduced model dimensions (`n_embd=768`, `n_head=12`, `n_layer=12`) for faster training and smaller model size, making it suitable for resource-constrained environments.
|
| 28 |
- Uses Byte-Pair Encoding (BPE) tokenizer trained on the same Wikipedia data.
|
| 29 |
|