Quantization made by Richard Erkhov.

Llama-3-Smaug-8B - GGUF

Model creator: https://huggingface.co/abacusai/
Original model: https://huggingface.co/abacusai/Llama-3-Smaug-8B/

Name	Quant method	Size
Llama-3-Smaug-8B.Q2_K.gguf	Q2_K	2.96GB
Llama-3-Smaug-8B.Q3_K_S.gguf	Q3_K_S	3.41GB
Llama-3-Smaug-8B.Q3_K.gguf	Q3_K	3.74GB
Llama-3-Smaug-8B.Q3_K_M.gguf	Q3_K_M	3.74GB
Llama-3-Smaug-8B.Q3_K_L.gguf	Q3_K_L	4.03GB
Llama-3-Smaug-8B.IQ4_XS.gguf	IQ4_XS	4.18GB
Llama-3-Smaug-8B.Q4_0.gguf	Q4_0	4.34GB
Llama-3-Smaug-8B.IQ4_NL.gguf	IQ4_NL	4.38GB
Llama-3-Smaug-8B.Q4_K_S.gguf	Q4_K_S	4.37GB
Llama-3-Smaug-8B.Q4_K.gguf	Q4_K	4.58GB
Llama-3-Smaug-8B.Q4_K_M.gguf	Q4_K_M	4.58GB
Llama-3-Smaug-8B.Q4_1.gguf	Q4_1	4.78GB
Llama-3-Smaug-8B.Q5_0.gguf	Q5_0	5.21GB
Llama-3-Smaug-8B.Q5_K_S.gguf	Q5_K_S	5.21GB
Llama-3-Smaug-8B.Q5_K.gguf	Q5_K	5.34GB
Llama-3-Smaug-8B.Q5_K_M.gguf	Q5_K_M	5.34GB
Llama-3-Smaug-8B.Q5_1.gguf	Q5_1	5.65GB
Llama-3-Smaug-8B.Q6_K.gguf	Q6_K	6.14GB
Llama-3-Smaug-8B.Q8_0.gguf	Q8_0	7.95GB

Original model description:

library_name: transformers license: llama2 datasets: - aqua_rat - microsoft/orca-math-word-problems-200k - m-a-p/CodeFeedback-Filtered-Instruction - anon8231489123/ShareGPT_Vicuna_unfiltered

Llama-3-Smaug-8B

Built with Meta Llama 3

This model was built using the Smaug recipe for improving performance on real world multi-turn conversations applied to meta-llama/Meta-Llama-3-8B-Instruct.

Model Description

Developed by: Abacus.AI
License: https://llama.meta.com/llama3/license/
Finetuned from model: meta-llama/Meta-Llama-3-8B-Instruct.

Evaluation

MT-Bench

########## First turn ##########
                   score
model             turn
Llama-3-Smaug-8B 1   8.77500
Meta-Llama-3-8B-Instruct 1   8.31250
########## Second turn ##########
                   score
model             turn
Meta-Llama-3-8B-Instruct 2   7.8875 
Llama-3-Smaug-8B 2   7.8875
########## Average ##########
                 score
model
Llama-3-Smaug-8B  8.331250
Meta-Llama-3-8B-Instruct 8.10

Model	First turn	Second Turn	Average
Llama-3-Smaug-8B	8.78	7.89	8.33
Llama-3-8B-Instruct	8.31	7.89	8.10

This version of Smaug uses new techniques and new data compared to Smaug-72B, and more information will be released later on. For now, see the previous Smaug paper: https://arxiv.org/abs/2402.13228.