Hugging Face's logo

RichardErkhov
/
hf-100_-_Llama-3.1-Spellbound-StoryWriter-70b-instruct-0.3-16bit-gguf

GGUF
Inference Endpoints
conversational
Model card Files Community
hf-100_-_Llama-3.1-Spellbound-StoryWriter-70b-instruct-0.3-16bit-gguf / README.md
RichardErkhov's picture
RichardErkhov
uploaded readme
b7ea62b verified
preview code
|
raw
history blame contribute delete
6.39 kB

Quantization made by Richard Erkhov.

Github

Discord

Request more models

Llama-3.1-Spellbound-StoryWriter-70b-instruct-0.3-16bit - GGUF

Name Quant method Size
Llama-3.1-Spellbound-StoryWriter-70b-instruct-0.3-16bit.Q2_K.gguf Q2_K 24.56GB
Llama-3.1-Spellbound-StoryWriter-70b-instruct-0.3-16bit.IQ3_XS.gguf IQ3_XS 27.29GB
Llama-3.1-Spellbound-StoryWriter-70b-instruct-0.3-16bit.IQ3_S.gguf IQ3_S 28.79GB
Llama-3.1-Spellbound-StoryWriter-70b-instruct-0.3-16bit.Q3_K_S.gguf Q3_K_S 28.79GB
Llama-3.1-Spellbound-StoryWriter-70b-instruct-0.3-16bit.IQ3_M.gguf IQ3_M 29.74GB
Llama-3.1-Spellbound-StoryWriter-70b-instruct-0.3-16bit.Q3_K.gguf Q3_K 31.91GB
Llama-3.1-Spellbound-StoryWriter-70b-instruct-0.3-16bit.Q3_K_M.gguf Q3_K_M 31.91GB
Llama-3.1-Spellbound-StoryWriter-70b-instruct-0.3-16bit.Q3_K_L.gguf Q3_K_L 34.59GB
Llama-3.1-Spellbound-StoryWriter-70b-instruct-0.3-16bit.IQ4_XS.gguf IQ4_XS 35.64GB
Llama-3.1-Spellbound-StoryWriter-70b-instruct-0.3-16bit.Q4_0.gguf Q4_0 37.22GB
Llama-3.1-Spellbound-StoryWriter-70b-instruct-0.3-16bit.IQ4_NL.gguf IQ4_NL 37.58GB
Llama-3.1-Spellbound-StoryWriter-70b-instruct-0.3-16bit.Q4_K_S.gguf Q4_K_S 37.58GB
Llama-3.1-Spellbound-StoryWriter-70b-instruct-0.3-16bit.Q4_K.gguf Q4_K 39.6GB
Llama-3.1-Spellbound-StoryWriter-70b-instruct-0.3-16bit.Q4_K_M.gguf Q4_K_M 39.6GB
Llama-3.1-Spellbound-StoryWriter-70b-instruct-0.3-16bit.Q4_1.gguf Q4_1 41.27GB
Llama-3.1-Spellbound-StoryWriter-70b-instruct-0.3-16bit.Q5_0.gguf Q5_0 45.32GB
Llama-3.1-Spellbound-StoryWriter-70b-instruct-0.3-16bit.Q5_K_S.gguf Q5_K_S 45.32GB
Llama-3.1-Spellbound-StoryWriter-70b-instruct-0.3-16bit.Q5_K.gguf Q5_K 46.52GB
Llama-3.1-Spellbound-StoryWriter-70b-instruct-0.3-16bit.Q5_K_M.gguf Q5_K_M 46.52GB
Llama-3.1-Spellbound-StoryWriter-70b-instruct-0.3-16bit.Q5_1.gguf Q5_1 49.36GB
Llama-3.1-Spellbound-StoryWriter-70b-instruct-0.3-16bit.Q6_K.gguf Q6_K 53.91GB
Llama-3.1-Spellbound-StoryWriter-70b-instruct-0.3-16bit.Q8_0.gguf Q8_0 69.83GB

Original model description:

base_model: unsloth/Meta-Llama-3.1-70B-Instruct-bnb-4bit language: - en license: apache-2.0 tags: - text-generation-inference - transformers - unsloth - llama - trl

Uploaded model

  • Developed by: hf-100
  • License: apache-2.0
  • Finetuned from model : unsloth/Meta-Llama-3.1-70B-Instruct-bnb-4bit

This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.