sayakpaul/mochi-lora

Mochi-1 Preview LoRA Finetune

Prompt

A black and white animated scene unfolds with an anthropomorphic goat surrounded by musical notes and symbols, suggesting a playful environment. Mickey Mouse appears, leaning forward in curiosity as the goat remains still. The goat then engages with Mickey, who bends down to converse or react. The dynamics shift as Mickey grabs the goat, potentially in surprise or playfulness, amidst a minimalistic background. The scene captures the evolving relationship between the two characters in a whimsical, animated setting, emphasizing their interactions and emotions

Model description

This is a lora finetune of the Mochi-1 preview model genmo/mochi-1-preview.

The model was trained using CogVideoX Factory - a repository containing memory-optimized training scripts for the CogVideoX and Mochi family of models using TorchAO and DeepSpeed. The scripts were adopted from CogVideoX Diffusers trainer.

Download model

Download LoRA in the Files & Versions tab.

Usage

Requires the 🧨 Diffusers library installed.

from diffusers import MochiPipeline
from diffusers.utils import export_to_video
import torch 

pipe = MochiPipeline.from_pretrained("genmo/mochi-1-preview")
pipe.load_lora_weights("CHANGE_ME")
pipe.enable_model_cpu_offload()

with torch.autocast("cuda", torch.bfloat16):
    video = pipe(
        prompt="CHANGE_ME",
        guidance_scale=6.0,
        num_inference_steps=64,
        height=480,
        width=848,
        max_sequence_length=256,
        output_type="np"
    ).frames[0]
export_to_video(video)

For more details, including weighting, merging and fusing LoRAs, check the documentation on loading LoRAs in diffusers.

Intended uses & limitations

How to use

# TODO: add an example code snippet for running this diffusion pipeline

Limitations and bias

[TODO: provide examples of latent issues and potential remediations]

Training details

[TODO: describe the data used to train the model]