Triangle104/Redemption_Wind_24B-Q5_K_S-GGUF

This model was converted to GGUF format from SicariusSicariiStuff/Redemption_Wind_24B using llama.cpp via the ggml.ai's GGUF-my-repo space. Refer to the original model card for more details on the model.

---

This model was undercooked on purpose. Target average loss value: 8.0

Mistral has blessed us with a capable new Apache 2.0 model, but not only that, we finally get a base model to play with as well. After several models with more restrictive licenses, this open release is a welcome surprise. Freedom was redeemed.

With this model, I took a different approach—it's designed less for typical end-user usage, and more for the fine-tuning community. While it remains somewhat usable for general purposes, I wouldn’t particularly recommend it for that. What is this model?

This is a lightly fine-tuned version of the Mistral 24B base model, designed as an accessible and adaptable foundation for further fine-tuning and merging fodder. Key modifications include:

ChatML-ified, with no additional tokens introduced.
High quality private instruct—not generated by ChatGPT or Claude, ensuring no slop and good markdown understanding.
No refusals—since it’s a base model, refusals should be minimal to non-existent, though, in early testing, occasional warnings still appear (I assume some were baked into the pre-train).
High-quality private creative writing dataset Mainly to dilute baked-in slop further, but it can actually write some stories, not bad for loss ~8.
Small, high-quality private RP dataset This was done so further tuning for RP will be easier. The dataset was kept small and contains ZERO SLOP, some entries are of 16k token length.
Exceptional adherence to character cards This was done to make it easier for further tunes intended for roleplay.

---

Use with llama.cpp

Install llama.cpp through brew (works on Mac and Linux)

brew install llama.cpp

Invoke the llama.cpp server or the CLI.

CLI:

llama-cli --hf-repo Triangle104/Redemption_Wind_24B-Q5_K_S-GGUF --hf-file redemption_wind_24b-q5_k_s.gguf -p "The meaning to life and the universe is"

Server:

llama-server --hf-repo Triangle104/Redemption_Wind_24B-Q5_K_S-GGUF --hf-file redemption_wind_24b-q5_k_s.gguf -c 2048

Note: You can also use this checkpoint directly through the usage steps listed in the Llama.cpp repo as well.

Step 1: Clone llama.cpp from GitHub.

git clone https://github.com/ggerganov/llama.cpp

Step 2: Move into the llama.cpp folder and build it with LLAMA_CURL=1 flag along with other hardware-specific flags (for ex: LLAMA_CUDA=1 for Nvidia GPUs on Linux).

cd llama.cpp && LLAMA_CURL=1 make

Step 3: Run inference through the main binary.

./llama-cli --hf-repo Triangle104/Redemption_Wind_24B-Q5_K_S-GGUF --hf-file redemption_wind_24b-q5_k_s.gguf -p "The meaning to life and the universe is"

or

./llama-server --hf-repo Triangle104/Redemption_Wind_24B-Q5_K_S-GGUF --hf-file redemption_wind_24b-q5_k_s.gguf -c 2048